| Neural Networks Learn To Remove Noise Without Seeing A Good Image |
| Written by Mike James |
| Sunday, 15 July 2018 |
|
Don't you just hate film grain? Except, of course, when you don't and set it as a special effect on your digital camera. NVIDIA now has a tool that will remove grain, noise and all manner of defects, and the amazing part is that it learned how to without seeing a clean image. How? From the point of view of machine learning this is almost magical. How can you teach a neural network to remove noise from an image when it is never given an example of what a clean image is. In other words, it isn't given the target it is aiming for. From the point of view of computational photography, or more generally signal processing, this is more than magic: We observe that under suitable, common circumstances, we can learn to reconstruct signals from only corrupted examples, without ever observing clean signals,and often do this just as well as if we were using clean examples. The idea is almost lunatic in its conception, but it has its roots going back to the use of the simple mean, or average, to estimate a value. The point is that, while the noise varies between images, the image remains constant. Thus, by training a network using nothing but noisy images, the network slowly learns to produce the average value of the images.
To illustrate the idea the team took images from the well known ImageNet and added random noise to them. They then trained a neural network using just the corrupted images. That is, the network was attempting to reproduce a corrupted image using a different corrupt image as its input. Over time it learned to produce the average, or probably more accurately the invariant part, of the image. Then, when shown a corrupt image it hasn't been trained on, it produces a cleaned up version.
This works much better than you could reasonably expect. See the video:
From simple Gaussian noise, the team went on to remove more complex types of corruption from the images. In particular, they had success removing a particularly difficult form of noise - Monte Carlo noise - that other methods have a tough time with. "We have shown that simple statistical arguments lead to surprising new capabilities in learned signal recovery; it is possible to recover signals under complex corruptions without observing clean signals, at performance levels equal or close to using clean target data. There are several real-world situations where obtaining clean training data is difficult: low-light photography (e.g., astronomical imaging),physically-based image synthesis, and magnetic resonance imaging."
This is a powerful idea - you can learn the noise structure just from degraded samples. As long as you can get multiple samples of the noise, you don't need the original. Is there something deeper going on here?
More InformationNoise2Noise: Learning Image Restoration without Clean Data Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala and Timo Aila, NVIDIA, Aalto University, and MIT. Related ArticlesUsing Super Seeing To See Photosynthesis Selfie Righteous - Corrects Selfie Perspective Computational Photography Shows Hi-Res Mars Seeing Buildings Shake With Software See Invisible Motion, Hear Silent Sounds Cool? Creepy? Extracting Audio By Watching A Potato Chip Packet
To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
| Last Updated ( Sunday, 15 July 2018 ) |
