Researchers have devised and tested a system that can search mobile phones in realtime in an efficient way to find targets in crowd-sourced images.

The mobile phone has revolutionized many things, photography in particular. Now everyone carries a high resolution digital camera with them and the world has never been so well recorded. Of course to access and make use of this data we need to be able to search it. Photos that are uploaded to the web are searchable but they don't provide a realtime view - they are the past.

Now suppose that it was possible to search the mobile phones directly and obtain a realtime view of what is happening. Once you think about this possibility you can immediately think up applications for it. Suppose a small child wanders off or is abducted - realtime picture search could use their accidental inclusion in other peoples photos to return an location and perhaps even a realtime track. You need to find the realtime whereabouts of a criminal, political dissident or escaped prisoner ... no problem just put out a worldwide photo search and wait. 

If you think that this is all crazy speculation then you might be surprised to hear that it has been tried out and it works.  Ardalan Amiri Sani and co-workers at Rice University in Texas describe in a recent paper a new system called Theia that can search mobile phones in an efficient way for target images.

All you have to do is download an app that connects to a server. A user of the service submits a query and the phone app performs a search on the photo's met data and the photo itself. For example you could specify a particular face and a particular background. To avoid overloading the phone the search is a two-stage operation. First a set of local predicates are derived from the query. These are simpler matching criteria that can be used to quickly select likely candidates for second stage of the search. If a photo is not rejected by the local predicates then it is uploaded to the Theia server and fully analyzed.

The key to not overloading the mobile phone with work is to arrange the predicates in to an order that rejects photos as quickly as possible. In addition a subsequent query is processed faster because of relevance locality. If a phone has returned a positive result then it is likely that phones that are geographically close will provide hits on similar queries. For example, if a phone has returned a match on a missing child then phones that are geographically close are more likely to produce further results.

The trial system was implemented as an Android application. The user has to specify how much processing power they are prepared to expend to achieve the result. That is, the user can specify a budget for the search. The testing of the system seems to suggest that you can host real-time photo searches on a smartphone without too much difficulty and, because of the parallel nature of the search results come back very quickly.

So what about privacy and security?

The designers suggest that people can opt in to store their photos in a Theia accessible folder - private photos can be stored elsewhere. Of course this raises the question of why you might be happy to hand over your photos and some of the processing power of your phone?

The designers of Theia don't discuss this aspect of the problem but real time photo search has commercial applications - finding just the right photo of a sporting event or news incident for example - and this could be used to fund the app. You might well start to see ads saying

"Install Theia and earn big money with your camera phone".

It is also likely that people could be encouraged to install such an app in response to a request to help find a missing child, say.

When you consider it carefully it could well be that real-time photo search is not just possible but positively attractive to the user.

More Information

Opportunistic Content Search of Smartphone Photos

arxiv.org/abs/1106.5568

If you would like to be informed about new articles on I Programmer you can either follow us on Twitter or Facebook or you can subscribe to our weekly newsletter.