Typing performance on Windows Phone 7 has been improved by using machine learning to change the size of the keys according to which one is most likely to be next.

Microsoft Research seems to be where all the best work is coming from these days as far as Microsoft is concerned. Now we have an account of how it helped improve the virtual or soft keyboard that Windows Phone 7 uses.

Accurate and fast keyboard input on a mobile phone is a well known problem. The standard approach to the problem is to simply use predictive text or dictionary look-up. This is good but still error prone. The example that given by the Microsoft team is “I ate the hummus” turns into “I ate the humans.” but my guess is that you can think of lots of real examples that were much more embarrassing.

To help improve accuracy the team decided to look at how hummus... sorry... humans interact with a virtual keyboard. They performed lots of experiments with a prototype phone and collected data on how users held the phone and how they typed - some used only their thumbs and proved that we really are distinguished from the apes by our opposable thumbs.

The improved performance was achieved using machine learning to map out the geometry of the keyboard space and relate it to what was being typed. Using statistical models of language patterns to implement a predictive text scheme and data gathered about where users tend to hit the keyboard it was possible to adjust the key areas to fit in with was was being typed. That is, the keyboard dynamically changes the virtual size of the keys to help the user hit the correct key.

Of course the display isn't changed it is just the target area corresponding to each key that is modified. In future versions of the software the speed of entry will also be taken into account in reconfiguring the keyboard. Two versions of the dynamic keyboard were implemented - simple dynamic where the target areas around each key were allowed to vary and anchor dynamic where the central areas of the keys were not moved. The anchored dynamic approach allowed the user to always input a character if they were accurate in pressing the key that they wanted.

Dynamic keyboard

Anchored dynamic keyboard

The results of testing suggest that anchored dynamic is best as it seems to make the keyboard respond as the user expects.

Another interesting approach is that the data collection program was converted into a game Text Text Revolution! that can be downloaded from the MarketPlace. Since its launch the game has collected more than 20 million touch points that can be used in training the machine learning algorithm.

Typing patterns as captured by the game Text Text Revolution.

Further reading

Download the full PDF of the academic paper on how the keyboard was created: Usability Guided Key-Target Resizing for Soft Keyboards

An account of the game:

Text Text Revolution: A Game That Improves Text Entry on Mobile Touchscreen Keyboards