Of course, the answer is "no, not yet". However, the difficulty of the task is lowered a bit by the restriction to multiple choice questions from a standardized 8th grade science exam. 

In case you missed it, the Allen Institute for Artificial Intelligence (AI2) is the creation of Paul Allen of Microsoft co-founder fame. While Bill Gates gives money away to cure the world's disease and poverty, Paul Allen has devoted a chunk of his money to "AI for the Common Good". 

The Institute has been working on a number of projects and Project Aristo is one of them. It is an "old school" symbolic information processor which gathers information and attempt to answer multiple choice science questions. What I mean by "old school" is that no neural networks were harmed during the building of the system. Instead a knowledge processing approach is used: 

• Natural language parsing and processing of a variety of knowledge sources into a structured, computable knowledge base
  
  
• Question understanding; transforming questions, diagrams, and answer options from exams into computable input
  
  
• Integration with diagram processing systems to go beyond text to successfully understand exam questions
  
  
• An array of solvers using various statistical and inference techniques to either select or generate the correct answer

A typical session with Aristo is something like:

Growing thicker fur in the winter helps some animals to

A) Hide from danger

B) Attract a mate

C) Find food

D) Keep warm

to which Aristo answers:

D) Keep warm. 

Artiso does reasonably well but AI2 wants it to do better so it has announced a Kaggle contest with $50,000 as the first prize for the system that gets most questions right - plus $20,000 for second place and $10,000 for third. 

The competition - The Allen AI Science Challenge - provides some training data, a validation data set and a sample submission. The training set has 2,500 questions and the validation set has 8,132 questions. The final test will use a data set of 21,298 questions. 

The really tough condition is:

"In order to be eligible for prizes, submitted models must run standalone without requiring internet connectivity (e.g., no use of runtime web searches or web APIs)."

Even IBM's Watson, the most successful question old school question answering system, couldn't get by with the occasional look at Wikipedia. 

Notice that the contest doesn't imply that you have to make use of anything similar to Aristo's knowledge processing. Some competitors are clearly thinking about using neural networks, although in this case the small size of the training set will make it difficult to find enough regularities to make it workable.

It will be interesting to see how it all ends up. 

The competition is open now and ends on 13th of Feburary 2016. Currenly there are 106 teams working on the problem. 

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info