An algorithm for face transfer
Written by Lucy Black   
Wednesday, 21 September 2011

With a little work you can take a photo of a face and paste it directly onto another face in a video. The result is that videos can now easily be made to lie.

Sometimes you just need a creative spark to notice that something can more or less be achieved with the technology we have to hand. Face detection/recognition is currently being used for all sorts of applications, but most of them are fairly obvious - security, photo tagging, photo cropping etc.

One really clever idea has been thought up by artist Arturo Castro and he has implemented it by borrowing on other peoples implementations of face detection and so on - after all one of the joys of software is that it is reusable. The idea is surprisingly simple. Using a face detection algorithm he was able to locate the position of a face in the video stream. Using this information he then used a photo to paste another face to the same location.


facepasteThe video Mao Tse-Tung never made.


The really clever part is that, as face detection locates the various regions of the face - eyes, mouth etc, this can be used on the photograph to distort the 2D face to the configuration of the face in the video. The face tracker software returns a mesh that can be used to mould the pasted face to fit.  What this means is that when the subject moves their mouth or eyebrows, the pasted image is distorted to fit. The result is that it appears that the pasted face is animated.

In Arturo's video below the pasted face is minimally blended with the live video and so there are artefacts that indicate that the video has been modified. In the second video by Kyle McDonald the blending has been improved and there is a much smoother transition between the pasted and original face. The overall effect is smoother but in many ways it seems to be more disturbing.


The original Arturo Castro video:



The smoother Kyle McDonald video:



While this is all great fun, there are some obvious things that this could be used for beyond just art installations. It clearly wouldn't take very much tweaking to make the result look naturalistic and then the possibility of producing videos impersonating people becomes all too easy. Now videos can lie just as easily as a photo.

More Information

Arturo Castro's website

The software used:


ofxFaceTracker addon



To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Twitter or Facebook or sign up for our weekly newsletter.



The New Android Compilers - Meet Jack And Jill

The new compiler toolchain, Jack and Jill, were included in the recent release of Android SDK 21.1, but only now has Google explained what they are are and what they do.

ENIAC Returns To Fort Sill

Almost 60 years after ENIAC was effectively broken up and scattered, a large chunk of it has been put back together. Although not restored to its original functionality, it is back in the location whe [ ... ]

More News

Last Updated ( Wednesday, 21 September 2011 )

RSS feed of news items only
I Programmer News
Copyright © 2014 All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.