You must have seen TV programs where the CSI person is asked to zoom in on face or worse the reflection of a face in the victim's eye. This is clearly impossible - or is it?

There is nothing worse than bad science and we all know what happens when you zoom in on a digital photo, and then zoom in again and again until some impossibly small patch of pixels becomes a full high resolution image of the suspect.

This Abstruse Goose cartoon sums up very nicely what would happen in real life:

Abstruse Goose

However, it now looks as if the joke is on us. A recent paper by Dr Rob Jenkins of the Department of Psychology at York University (UK) has managed to prove that you can get useful images of faces from the reflections in eyes. It really is as simple as zooming in. The catch is that the experiments were done with a 39 mega pixel camera - even so the actual images were low resolution.

The reason it works, and it seems to be reliable with a 84% accuracy, is that we are good at recognising faces even with low quality images. The key factor is that the test subject has to know the face in the low resolution image. It seems that we are very good at identifying a known face and not so good at matching a low res image to a set of possible hi-resolution photos including the face. 

For example, even though the photo below is just 16x20, most people will be able to recognize who it is of:

In the experiment a number of people were photographed with a "bystander" in a position so that a reflection of their face would be captured in the eye. The resulting extracted image of the reflection in the eye was only 27x36 and then rescaled using bicubic interpolation to 400x240 or bigger and enhanced using standard PhotoShop operations to normalize the contrast and brightness.

Test subjects were able to match faces using the low resolution images but the important result was that if the subject knew the person in the photo then recognition went up to 90% with false positives down at 10%. 

The paper concludes with a discussion of how this might all be used and improved. For example, could two reflections be used to derive depth data and even images of people not actually visible to the subject. 

If you think that the high resolution images are unlikely to be available it is worth pointing out that the Nokia Lumia 1020 has a 41Mpixel camera and no doubt soon this will be the norm. As the paper points out it might not be the quality of the corneal reflection that matters but how well the person trying to recognize the face knows the target. 

So the next time you appear in a photo consider the fact that a simple procedure might reveal who you are with. 

Can we do better? 

Clearly there is scope for some computational photography to improve the quality of the extracted images.

Is there an optimal rescaling algorithm?

Can the depth information be used to construct a better model of the face?

Can the whole task be automated with face extraction and recognition. 

The answers to these questions might take TV crime investigation into the real world.