This is not about tracking Android faces, but human faces. It isn't face recognition, but it is just as useful for many applications where you just need to know where a face is and if it is smiling, say.

Update: The face tracker runs on the client only - no server needed!

This is a really nice new Android facility. The only worrying part about it is that it is being delivered as a Google Play Service.

This is the general worry about becoming reliant on any Google Play Service rather than being specific to this API. Google has been moving increasingly large chunks of the Android system to Google Play. The advantage to the user is that they don't have to rely on the hardware manufacturer to provide updates - Google Play does the job. The downside is that the Android hardware manufacturer has to strike a deal with Google to use Play Services and not all do. This means you have to check that you are working with a "real" Android device.

The new API is interesting because it isn't a face recognition system and so there are fewer privacy issues.

It detects a face when it first appears, assigns it an id and then tracks it as it moves. When the face goes out of scene the id is destroyed and if it moves back into view it is assigned a new id and there is no way to associate it with its previous appearances. 

At its most basic, the API will tell you where in an image or video there are faces. It provides a bounding box complete with id and this moves as the face moves. It can detect faces at a very wide range of orientations and it can report the orientation back to you, making it possible to work out what the face is looking at. For example, at the most basic, it can tell if the face is looking at the Android device.

You can also ask for a list of landmark features for each face -  bottom of mouth, left cheek, left ear, left ear tip, left eye, left mouth, base of nose, right cheek, right ear, right ear tip, right eye or right mouth. What you could do with the locations of these isn't clear, but presumably you could use them to warp another face into position or something similar

Finally, you can ask for the probability of an activity - left eye open, right eye open and smiling. As the Android developer blog points out, you could use this to create an app that only takes a photo when everyone was looking a the camera and smiling. But now this idea has been suggested don't bother because a lot of people will have beaten you to it. 

The good news is that it all looks easy to use. The documentation is good and there are sample apps on GitHub. You can feed it still images or videos, but if it can't keep up it just drops frames. You basically set up a processing pipeline from the camera through the face tracker and once set up it just keeps processing:

Update:

Originally we said that the client side code was open source - it isn't. Google corrected us and pointed out that some examples were open source but the bulk of the client implementation is not. 

The reason for this is that there is no server involved!

The face detector runs completely on the client device and doesn't contact any Google Play server.

Originally we speculated on what service quotas or charges there might be - taking for granted that a server had to be involved in such heavy processing. Clearly this isn't the case and no quotas or charges are involved or pending for this version of the API, as has been confirmed by Google. 

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info