Since Photoshop you haven't been able to trust a still photograph. Now it seems you can't trust a video. Real-time face capture and reenactment allows one face to control another so that you can make anyone say anything with any expression you care to pull. 

There have been facial reenactment systems before, see Create Your Favourite Actor From Nothing But Photos, but this is the first time that there is the ability to put words into another person's mouth and an expression onto their face.

The system works by taking a video stream featuring the target person and modifying it based on the facial expression of a source actor. So, if the source actor smiles, the person in the target video stream smiles - even if they didn't in the actual video stream.

The easiest way to fully understand what this means and what potential it might have is to watch the following video. Notice that the expressions of the source actor appear on the target in the reenactment video.

What is impressive is that it works in real-time with a cheap camera. The way that it works is fairly straightforward - detect the face and the features in the source actor's video and in the target video, then distort the target's facial features to match thos of the source actor. Easy to explain but very difficult to do well. 

Given some of the subjects used in the video as the test subjects - President Putin, Donald Trump, President Obama and so on -  you can start to imagine what mischief the system could be used for.

Want to change a polical speech?

Very easy.  

Of course, there are more honestly useful applications for this sort of technology. You could redub films in different languages and have the actors seem to say the new dialog. It might even be better to get the actors to do the job without saying a word in any language and then add the dialog later. You could also correct actors errors in expensive staged stunts.  

It used to be the folk law that the camera never lies.

Now video cameras can join still cameras in the role of excellent liars. 

More Information

Face2Face: Real-time Face Capture and Reenactment of RGB Videos Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, and  Matthias Nießner 

Related Articles

Create Your Favourite Actor From Nothing But Photos 

Watching Paint Dry - GPU Paint Brush

Better 3D Meshes Using The Nash Embedding Theorem

3-Sweep - 3D Models From Photos

Time-Lapse Videos From Internet Photos

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info