Graphics techniques are becoming so sophisticated that it really does seem like magic is being done. Now a team of researchers has implemented a way of constructing a 3D model of a well known actor and using it to create scenes they never played in.

If you always wanted to see John Wayne play the lead in The Good, the Bad and the Ugly you might not have to wait much longer. 

A University of Washington team has essentially reversed engineered what makes an actor recognizable as that actor, or as the title of the paper puts it "What Makes Tom Hanks Look Like Tom Hanks". 

The model is first created using photos. The method relies on facial recognition to locate the standard points that make up a face. The face is then warped to place the standard points in the same orientation so that all of the photos are reduced to face-on shots. Then they are wrapped around a standard 3D model customized to give the required distances between the standard points. The warped photos are then used to construct a texture to overlay the 3D model to make it look natural and more like the target. 

This amazing enough, but the next stage is to analyse a video of the "driver", i.e. to work out the deformations in the puppet needed to make it follow the driver. This is the real innovation - working out how to convert the movement of one person into those of the actor.

To quote the paper:

"Our answer to the question of what makes Tom Hanks look like Tom Hanks is in the form of a demonstration, i.e., a system that is capable of very convincing renderings of one actor believably mimicking the behavior of another. Making this work well is challenging, as we need to determine what aspects are preserved from actor A’s performance and actor B’s personality. For example, if actor A smiles, should actor B smile in the exact same manner? Or use actor B’s own particular brand of smile? After a great deal of experimentation, we obtained surprisingly convincing results using the following simple recipe: use actor B’s shape, B’s texture, and A’s motion (adjusted for the geometry of B’s face)."

So it seems that it is sufficient to use the movements generated by the driver and simply modify them to take account of the geometry of the actor's face.

Although the researchers don't draw any conclusion from this, it suggests that the character of an actor comes about from displaying a universal human facial expression using the physiology they happen to have. So much for acting...

Take a look a the video to see if you find the techniques convincing:

The possiblities are clear, endless, and slightly frightening if you happen to be an actor. 

More Information

What Makes Tom Hanks Look Like Tom Hanks. International Conference on Computer Vision (ICCV)

Supasorn Suwajanakorn, Steven M. Seitz, Ira Kemelmacher-Shlizerman. 

Related Articles

Watching Paint Dry - GPU Paint Brush

Better 3D Meshes Using The Nash Embedding Theorem

3-Sweep - 3D Models From Photos

Time-Lapse Videos From Internet Photos

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook, Google+ or Linkedin. 

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info