An algorithm for face transfer
Written by Lucy Black   
Wednesday, 21 September 2011

With a little work you can take a photo of a face and paste it directly onto another face in a video. The result is that videos can now easily be made to lie.

Sometimes you just need a creative spark to notice that something can more or less be achieved with the technology we have to hand. Face detection/recognition is currently being used for all sorts of applications, but most of them are fairly obvious - security, photo tagging, photo cropping etc.

One really clever idea has been thought up by artist Arturo Castro and he has implemented it by borrowing on other peoples implementations of face detection and so on - after all one of the joys of software is that it is reusable. The idea is surprisingly simple. Using a face detection algorithm he was able to locate the position of a face in the video stream. Using this information he then used a photo to paste another face to the same location.

 

facepasteThe video Mao Tse-Tung never made.

 

The really clever part is that, as face detection locates the various regions of the face - eyes, mouth etc, this can be used on the photograph to distort the 2D face to the configuration of the face in the video. The face tracker software returns a mesh that can be used to mould the pasted face to fit.  What this means is that when the subject moves their mouth or eyebrows, the pasted image is distorted to fit. The result is that it appears that the pasted face is animated.

In Arturo's video below the pasted face is minimally blended with the live video and so there are artefacts that indicate that the video has been modified. In the second video by Kyle McDonald the blending has been improved and there is a much smoother transition between the pasted and original face. The overall effect is smoother but in many ways it seems to be more disturbing.

 

The original Arturo Castro video:

              

 

The smoother Kyle McDonald video:

 

 

While this is all great fun, there are some obvious things that this could be used for beyond just art installations. It clearly wouldn't take very much tweaking to make the result look naturalistic and then the possibility of producing videos impersonating people becomes all too easy. Now videos can lie just as easily as a photo.

More Information

Arturo Castro's website

The software used:

​FaceTracker

ofxFaceTracker addon

Openframeworks​

 

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Twitter or Facebook or sign up for our weekly newsletter.

 

Banner


Learning To Be A Computer
29/10/2014

We have recently seen how much neural networks can learn, but the latest example is very strange. Google has succeeded in teaching a neural network to be a computer simply by showing it programs and w [ ... ]



Last Call for SharePoint Conference
11/11/2014

 This year’s SharePoint Connect conference is almost here, with just a week left for new registrations.


More News

Last Updated ( Wednesday, 21 September 2011 )
 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2014 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.