If you were looking for another reason not to get a smartwatch, researchers at ECE Illinois have created an app that can work out what you are typing just from your hand movements.

The app is only available for the Samsung Gear Live smartwatch, but other devices will leak data in the same way. 

Smartwatches generally have a good range of sensors, including a gyro and accelerometer. These are supposed to be used for tracking hand gestures and are generally easily available to apps. For example an app that claims to be just a pedometer could use its access to the gyro and accelerometer to work out what you are typing.,

The movement data can be use to derive timing and 2D displacement and hence which key you are hitting. For example the left wrist moves further for a T than an F.  If you think that this is unlikely because you do not move your hands much as you type then take a look at the following video:

The point is that the movements are small but the sensors are up to the job of tracking them. Combine this with a little machine learning and the inherent redundancy of language and you have a predictor that is more accurate than you might expect. 

The project is called Motion Leaks through Smartwatch Sensors, or MoLe, and as Associate Professor Romit Roy Choudhury explains: 

"Sensor data from wearable devices will clearly be a double-edged sword. While the device's contact to the human body will offer invaluable insights into human health and context, it will also make way for deeper violation into human privacy. The core challenge is in characterizing what can or cannot be inferred from sensor data and the MoLe project is one example along this direction.”

It is suggested that a possible solution to the problem is to make the sensors less precise by lowering the sampling rate from the usual 200Hz to something below 15Hz. Of course this would make some gesture recognition programs unable to function. 

The researchers also point out that their methods are not perfect. The app only works if the user types in the standard way, i.e. a home keys based style. There are also problems with special characters and the space bar. However, with improved techniques these problems could be overcome and the app could in principle reveal everything you type with a reasonable accuracy.

One problem that is difficult to find a solution to is that the smartwatch only tracks one hand - the keys pressed by the other hand are usually much harder to detect. 

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info