We reported on Eulerian Magnification about a year ago. It is a very simple technique that can be used to let you see things that are beyond the range of normal human vision. Now the MIT research group has released its software for us all to use.
Eulerian magnification is a signal processing technique that can extract data in a video stream that is certainly there but cannot normally be noticed by a human observer. The problem for the human is that the eye can't keep track of the changes in color at a particular point in an image. This is partly because the eye isn't very good at tracking temporal changes and partly because it also needs to track the movement of the point of interest in the visual field.
This might be hard for a human, but for a computer it is much easier. A group of researchers at MIT CSAIL and Quanta Research Cambridge, MA has applied very fundamental image processing techniques - spatial decomposition and temporal filtering - to standard video to show details that are normally very hard, if not impossible, to see. It works by tracking color variations in time at fixed locations in the image.
The amazing fact is that doing this reveals a world of things in a standard video that you could see even if you had super vision and it doesn't require special hi-resolution video cameras to do the job. It can even process videos that have been uploaded to You Tube but the results do depend on the quality of the video.
You can see it in action in the following videos:
If you point it at a human then you can see the blood flow as a color variation in the skin. This has obvious uses in medicine and monitoring. It can also be used to magnify very small movements and make them visible. Viewing any of the examples is very much like seeing a time lapse video for the first time - you see a world that you didn't know existed.
You can download the software and run it using MatLab. It isn't clear if it works with Octave, the open source matrix language very similar to MatLab, but it shouldn't be difficult to modify. If you don't want to run the software you can also upload your own videos to the Quanta Research website and let them process it.
The MIT team plans to release a smart phone app that does the same job. Clearly this is also one that would make Google Glass and interesting device!
The only worrying negative point about the whole idea is that the main web site indicates that MIT has applied for a patent. It isn't clear exact ly which aspects of the method are subject to the patent. The software is available for non-commercial use and while the source is available - it being MatLab code there isn't really any other way to make it available - it isn't open source.
As programmers we often think that users are overly sensitive about their data. What could it hurt to allow the collection of location data, for example. Here is a short video from the ACLU that might [ ... ]