Too Good To Miss: Back in June 2021 we looked at a way to make a photo more than a photo and not just inside your head. Looking at the photo of a waterfall you are often convinced the water is moving.

Now researchers at the University of Washington have developed a new deep learning method that can animate portions of a photograph, turning it into a video - and the result is just the cascade you would expect.

Yes, we have another example of what can be done with a GAN and it was presented at the 2021 Conference on Computer Vision and Pattern Recognition. The method, which has the  can animate flowing material such as waterfalls, smoke, or clouds. The technique developed at the University only needs a single photo of a waterfall to create the animation.

Lead author of the paper, Animating Pictures with Eulerian Motion Fields, Aleksander Hołyński, a doctoral student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington commented:

“A picture captures a moment frozen in time. But a lot of information is lost in a static image. What led to this moment, and how are things changing? What’s special about our method is that it doesn’t require any user input or extra information. All you need is a picture. And it produces as output a high-resolution, seamlessly looping video that quite often looks like a real video.”

As explained in the video, the method used to create a realistic looping video from a still images relies on the observation that this type of natural motion can be convincingly reproduced from a static Eulerian motion description, i.e. a single, temporally constant flow field that defines the immediate motion of a particle at a given 2D location. It uses an image-to-image translation network to encode motion priors of natural scenes collected from online videos, so that for a new photo the corresponding motion field can be synthesized. 

To avoid the holes and blank spaces that would otherwise occur in the video the researchers used a novel video looping technique that flows features both forward and backward in time and then blends the results.

For more examples and more importantly for the code which is "coming soon", go to the project's website.

More Information

Animating Pictures with Eulerian Motion Fields by Aleksander Holynski, Brian Curless, Steven M. Seitz, Richard Szeliski

Related Articles

GANs Create Talking Avatars From One Photo

Using GANs For Underwater Color Images

GANPaint: Using AI For Art

GANalyze - What Makes Pictures Memorable?

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info