Animating Flow In Still Photos
Written by David Conrad   
Sunday, 27 June 2021

Looking at the photo of a waterfall you are often convinced the water is moving. Now researchers at the University of Washington have developed a new deep learning method that can animate portions of a photograph, turning it into a video - and the result is just the cascade you would expect.

Yes, we have another example of what can be done with a GAN and it was presented at the 2021 Conference on Computer Vision and Pattern Recognition. The method, which has the  can animate flowing material such as waterfalls, smoke, or clouds. The technique developed at the University only needs a single photo of a waterfall to create the animation.

waterfall

Lead author of the paper, Animating Pictures with Eulerian Motion Fields, Aleksander Hołyński, a doctoral student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington commented:

“A picture captures a moment frozen in time. But a lot of information is lost in a static image. What led to this moment, and how are things changing? What’s special about our method is that it doesn’t require any user input or extra information. All you need is a picture. And it produces as output a high-resolution, seamlessly looping video that quite often looks like a real video.”

 

 

As explained in the video, the method used to create a realistic looping video from a still images relies on the observation that this type of natural motion can be convincingly reproduced from a static Eulerian motion description, i.e. a single, temporally constant flow field that defines the immediate motion of a particle at a given 2D location. It uses an image-to-image translation network to encode motion priors of natural scenes collected from online videos, so that for a new photo the corresponding motion field can be synthesized. 

To avoid the holes and blank spaces that would otherwise occur in the video the researchers used a novel video looping technique that flows features both forward and backward in time and then blends the results.

For more examples and more importantly for the code which is "coming soon", go to the project's website.


More Information

Animating Pictures with Eulerian Motion Fields by Aleksander Holynski, Brian Curless, Steven M. Seitz, Richard Szeliski

Related Articles

GANs Create Talking Avatars From One Photo

Using GANs For Underwater Color Images

GANPaint: Using AI For Art

GANalyze - What Makes Pictures Memorable?

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner


Introduction to Machine Learning with Scikit-Learn
14/07/2021

A free course on the fundamentals of Machine Learning with Python, taught by Kevin Markham founder of Data School, helps you ease your way into ML and scikit-learn, one of the best-known libraries for [ ... ]



Learn Python with HackInScience
02/07/2021

HackInScience is a code challenge site that is mostly oriented towards learning than competing with others for a top ranking despite the presence of a Leaderboard. It's a playground of fun Python [ ... ]


More News

square

 



 

Comments




or email your comment to: comments@i-programmer.info

 

Last Updated ( Sunday, 27 June 2021 )