With a little work you can take a photo of a face and paste it directly onto another face in a video. The result is that videos can now easily be made to lie.

Sometimes you just need a creative spark to notice that something can more or less be achieved with the technology we have to hand. Face detection/recognition is currently being used for all sorts of applications, but most of them are fairly obvious - security, photo tagging, photo cropping etc.

One really clever idea has been thought up by artist Arturo Castro and he has implemented it by borrowing on other peoples implementations of face detection and so on - after all one of the joys of software is that it is reusable. The idea is surprisingly simple. Using a face detection algorithm he was able to locate the position of a face in the video stream. Using this information he then used a photo to paste another face to the same location.

The video Mao Tse-Tung never made.

The really clever part is that, as face detection locates the various regions of the face - eyes, mouth etc, this can be used on the photograph to distort the 2D face to the configuration of the face in the video. The face tracker software returns a mesh that can be used to mould the pasted face to fit.  What this means is that when the subject moves their mouth or eyebrows, the pasted image is distorted to fit. The result is that it appears that the pasted face is animated.

In Arturo's video below the pasted face is minimally blended with the live video and so there are artefacts that indicate that the video has been modified. In the second video by Kyle McDonald the blending has been improved and there is a much smoother transition between the pasted and original face. The overall effect is smoother but in many ways it seems to be more disturbing. 

The original Arturo Castro video:

Faces from arturo castro on Vimeo.

The smoother Kyle McDonald video:

Face Substitution from Kyle McDonald on Vimeo. 

While this is all great fun, there are some obvious things that this could be used for beyond just art installations. It clearly wouldn't take very much tweaking to make the result look naturalistic and then the possibility of producing videos impersonating people becomes all too easy. Now videos can lie just as easily as a photo.

More Information

Arturo Castro's website

The software used:

​FaceTracker

ofxFaceTracker addon

Openframeworks​

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Twitter or Facebook or sign up for our weekly newsletter.