Computational photography is approaching the status of magic. Imagine amassing photos from all round the world taken by different people at different times and putting them together to make time-lapse videos of popular locations. You don't have to imagine because Google together with the University of Washington has done it. 

Traditionaly time-lapse videos are obtained by fixing a camera to a spot and taking one photo every few hours or even days. If you want a time-lapse over a few years then you either have to be patient or have had to thought about it a long time ago. However, people wander about and take photos of popular landmarks all the time. Often taking them from the same viewpoints. This provides a huge back catalog of potential frames for a time-lapse video that means you can see how something has changed over time.

The problem, of course, is that these potential frames are not taken by a camera from a fixed position and simply putting them together in time stamp order would produce a video that was impossible to watch.

Researchers at the University of Washington and Google have applied a number of computational photographic techniques to effectively stabilize the images in the videos. They call the overall technique "time-lapse mining". 

First they started off with a staggering 86 million timestamped and geotagged photos from around the world. Their system automatically works out which locations have enough images and creates a time-lapse video for them. 

You can imagine the sort of steps that are needed in the processing of the images to create a time-lapse but consider the attention to detail that is necessary to create a high quality result. Given a set of photos taken at a given location the first task is to use SIFT features to group photos with similar content. For example, you might have been at the location of the well known monument but you might have been taking photos of the people you were with or of the crowd. Next the set is reduced to just include photos that are taken from the same viewpoint. 

The next step is to compute a depth map from the viewpoint of a reference image. This depth map is used to warp all the other images into the same reference frame. Next each image is normalized to account for differences in exposure and lighting. Finally an in-painting algorithm was used to fill in missing pixels. 

This is a lot of computation but it seems to work: 

In total the system created 10,728 time-lapse videos. To quote from the paper (to be presented at this year's SIGGRAPH):

"The scale and ubiquity of our mined time-lapses creates a new paradigm for visualizing global changes. As more photos become available online, mined time-lapses will visualize even longer time periods, showing more drastic changes."

It seems that the internet really is collecting data on the state of the entire planet. Why stop at still images? There are millions of surveillance cameras that could provided complete video sequences. Perhaps soon it will be possible to prove what you did and where you were in the past.

Visualizations of the data help us see what is there, but it doesn't get us close to a practical use for the data. 

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info