Almost Still Photos - Free Cliplet App From MS Research
Written by David Conrad
Thursday, 08 March 2012
Microsoft Research has just invented "almost" still photography and you can try it out for yourself with a free to download app that converts video clips into photos with just a hint of movement.
Still photography is an art form but just occasionally you can't help feel that just a little motion would make things better. The impressive picture of the sea breaking on the shore might be even more impressive if one or two waves actually did some breaking. As the team from Microsoft Research puts it:
A still photograph is a limited format for capturing moments that span an interval of time. Video is the traditional method for recording durations of time, but the subjective "moment" that one desires to capture is often lost in the chaos of shaky camerawork, irrelevant background clutter, and noise that dominates most casually recorded video clips. This work provides a creative lens used to focus on important aspects of a moment by performing spatiotemporal compositing and editing on video-clip input. This is an interactive app that uses semi-automated methods to give users the power to create "cliplets"—a type of imagery that sits between stills and video from handheld videos.
It sounds like an appealing idea. Now with the help fo Microsoft Research, some clever image processing and the resulting Cliplet application, you can take a video clip and convert it into a Cliplet. All you have to do is download the app and then drag-and -drop a video clip onto the work area. Next you simply outline areas that you want to be still or moving. The idea is that you can map which parts of the video display for which relative time periods in the finished cliplet. So you can pick out one part of the video to move against a frozen background, say, or you can create short loops of motion, or ... To be honest it all becomes much more obvious if you just watch the video of how it is achieved:
The clever algorithms do the work of segmenting the video, stabilizing the images and putting the whole thing together without the joins showing.
Is this a good idea?
There is never any problem with trying to reinvent photography and, even if the current examples aren't impressive, it doesn't mean that a creative person couldn't do something amazing with the algorithms we give them. However, on the current showing the cliplet either seem to be a novelty, a bit like birthday cards that play a tune, or they seem slightly spooky, like the example of the girl whose hair and a tiny piece of clothing move in the breeze.
Perhaps all it needs is a genius to raise it up to high art, but if it is going to catch on I think the process will have to be made even easier than it currently is. Perhaps we need a camera that takes cliplets directly.
The first issue of a new open access journal with the title The Art, Science, and Engineering of Programming is now available on arxiv.org. It is published under the auspices of AOSA in conjuncti [ ... ]