Your phone can now be hijacked by malware to take photos of your surroundings and allow an attacker to build a 3D model from the data. Once built, the model lets an attacker explore to find weaknesses, gather information and steal secrets.

This sounds like your worst nightmare - but it is not fiction.

The US Naval Surface Warfare Center working with Indiana University has just worked out how to infect a mobile phone with a Trojan that can take photos without you knowing anything about it and send sensor data back to a server. The data are used to construct a 3D model which can be used not only to perform the reconnaissance necessary to break in, but also to steal confidential information such as bank details.

Given the 3D model on the left, the thief can select an object and see a higher resolution photo of it - in this case a cheque complete with details.

What is really interesting about the technique is that it is a clever implementation of existing cutting edge methods, combined with a lot of refinement. For example, there are procedures which will turn a collection of 2D photos into a 3D model but, at the moment, these are too demanding to run on a phone. So the only alternative is to send the photos to a server and do the processing there. This raises the problem of the large amount of data that needs to be sent over a limited connection. The solution is to simply pick the best photos from the ones taken. The app examines each photo and throws away any that are blurred, too dark, or repeats of photos already selected. The accelerometer is used to group photos into sets that are likely to be of the same view. Then these are processed to select the best of the set which is sent to the server for processing.

A collection of opportunistically taken photos is used to synthesize a full 3D model that is much easier to interpret.

It also turns out that you don't need super resolution photos to construct reasonable 3D models.  It was found that 1-megapixel images were good enough, and this too reduces the bandwidth needed.

A 3D model constructed using 1 Megapixel images resolves all the objects you need to see.

The app was implemented on an Android 2.2 system, but clearly it could have been engineered to work with iOS or any phone OS. However, the techniques for getting it to work with the Android are interesting. For example, to stop the camera showing the user the picture just taken all that was necessary was to pass a null display surface. A more difficult problem was stopping the fake click that all digital cameras make. Apparently you can't turn this off without root permissions. The solution was to turn down the volume while the photo was being taken - the user would just think it was a connection glitch. Too easy.

The app seems to have worked well in tests in which individuals were given the infected phone and told to make use of it in a normal way.  The researchers were able to reconstruct good 3D models in most cases.

You might be wondering why bother with the 3D model rather than just looking at the photos coming back from the phone?

The reason is that sifting though all those jumbled poor images is difficult compared to just exploring a 3D representation of, say, the user's office. You can pan around, take a look at the calender, peer at the documentation on their desk and so on.  Again experiments proved that potential thieves found the 3D model much easier to use then the collection of photos. In short, the 3D model acts as an organizing principle for the disorganized photos and provides a 3D map that can be used to find potential entry points and valuables to steal.

The team goes on to suggest improvements that might be possible in the future as phones become more powerful, such as using AI software to pick out the really interesting details. They also suggest ways that such attacks might be blocked - perhaps the easiest is just making sure that the camera click sound can't be muted. However, as we know, to every defense there is an offense and at best we just slow the attack down.

At the end of the day, the work is both scary and encouraging. The fact that we carry with us a device that can be used to build a 3D model makes us less secure, physically less secure. The same facilities and the potential for processing also promise some interesting positive applications whereby anyone with a mobile phone can provide a 3D model that could be used by life saving (or just useful) robots.