Watson wins Jeopardy! - trick or triumph
Written by Mike James   
Friday, 18 February 2011
Article Index
Watson wins Jeopardy! - trick or triumph
Getting it right - or wrong

IBM's Watson finished the recent contest between man and machine with $77,147, compared to $24,000 for Ken Jennings and $21,600 for Brad Rutter, another top Jeopardy! champion. This is amazing and there is plenty of talk of the "day of the machine". But wait! Watson doesn't think or understand anything. It's not even a question-answering machine - but an answer-questioning machine which is perhaps a whole lot simpler. So is it a triumph of machine over man or of publicity over fact?

 

final

 

IBM has pulled off a triumph of publicity, if not AI (artificial intelligence), in creating a machine that can beat seasoned players at the game of Jeopardy!. As well as the publicity from the show, IBM made campus appearances and generally generated enthusiasm among students. The result is that now IBM looks like a cool company to work for and has moved from grey and outdated to being up there with Google, Twitter and Facebook. It has also raised the public perception of AI and what computers can do beyond word processing and browsing the web. If you are semantic engineer or an expert in machine learning then you can expect to be in more demand in the future. 

For this at least IBM deserves thanks but is Watson AI or is it a trick?

How AI works

AI is a strange subject because by its nature it is doomed to be perceived has having failed. Consider just how it works. A human does something like play chess or answer questions on Jeopardy! and we immediately credit the behaviour as an example of intelligence. Intelligence is what humans and occasionally some animals are assumed to have without having to prove anything much apart from engaging in the activity.

Now compare this to a chess playing program. At first it looks impressive, especially when it wins, but when you examine how it does the job you discover that its an easy to follow algorithm. You can find out exactly how a chess program works in terms of searching and evaluating the next move in terms of what might happen n moves on. Even the most sophisticated variations on the algorithm seem simple and crude. Even though the program can beat a grand master, like IBM's previous AI stunt with Deep Blue, it just doesn't seem to be made of the same stuff as human intelligence.

If you ask what would be required of an AI program to  impress you enough to be worth calling intelligent then what you end up demanding is a large slice of "mystery". Every time AI succeeds in reducing a human behaviour to an algorithm it immediately changes from intelligence to a machine procedure. With this in mind it is time too look at Watson.

 

icon

Watson - a statistical approach to AI

The reason why Watson is impressive is that to complete the task it has to bring together a range of separate AI techniques. It has to understand natural language well enough to process the question and formulate an answer. But first notice that Jeopardy! is the reverse of a standard quiz show. It provides the answers and the contestants have to formulate the questions. There are also elements of question selection and betting to be mastered but the main AI task is to formulate a question given the answer.

This task would seem to need complete understanding at a very human level but you would probably be wrong. The key idea in most of the really successful applications of AI in the last few years has been statistics.The statistical approach to AI may have produced many successes - Google Translate for example - but many regard it as being unsatisfying in the sense described earlier.

Suppose we have the very simple AI task of writing a program to guess the number you have just thought of - yes it's silly but illustrates the point.  A valid AI approach would be to try to use subtle hints from your psychology and recent experiences to formulate a model of your cognitive functions and so work out your most likely numeric selection. The statistical approach would simply get you to play the game millions of times and work out statistically what number was most likely. The statistical approach to language understanding has only recently become possible because of the huge amount of language data that the web provides.

What Watson does is to take the input question and use some syntax analysis on it, but not with the intention of understanding the question - just to split it into functional fragments.  These are then used to discover what the question is by a statistical process of searching the knowledge base for something that has entries that correspond to the data in the question. When an entity is found that matches the features of the question then it is considered as a possible answer. A set of heuristic confidence levels are computed and the final answer is constructed from the entity that has the best confidence level. The confidence level is also used to determine if Watson should buzz in or not.

For example suppose the question was:

Category: "Rap" Sheet

Clue: This archaic term for a mischievous or annoying child can also mean a rogue or scamp.

Then the clue is processed to create fragments such as "mean" "rogue or scamp" these are then looked up in the knowledge base and if an entry has "rogue or scamp" or "mean(s)" and "rogue or scamp" then the entry is the possible subject of the answer.



Last Updated ( Monday, 19 March 2012 )
 
 

   
RSS feed of all content
I Programmer - full contents
Copyright © 2014 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.