Douglas Lenat was a proponent of symbolic, as opposed to statistical machine learning and the creator of Cyc, a rule-based approach to AI. After devoting himself for over four decades to the task of representing all human knowledge as a set of logical, commonsense rules, he has died at the age of 72. 

Having completed three degrees -  bachelor’s degrees in both math and physics and a master’s in applied mathematics — in three years, at the University of Pennsylvania, Lenat moved to Stanford to study artificial intelligence. His received his PhD in 1976 with a thesis outlining the Automated Mathematician (AM) described on Wikipedia as of the earliest successful discovery systems that attempts to discover new scientific concepts or laws. Not only was his thesis published by McGraw Hill, it is one of two included in Knowledge-based Systems in Artificial Intelligence (1982), AM led to Lenat being awarded the IJCAI Computers and Thought Award in 1977.

Douglas Lenat in 2017

AM worked by generating and modifying short Lisp programs which were then interpreted as defining various mathematical concepts for example, a program that tested equality between the length of two lists was considered to represent the concept of numerical equality, while a program that produced a list whose length was the product of the lengths of two other lists was interpreted as representing the concept of multiplication. The system had elaborate heuristics for choosing which programs to extend and modify, based on the experiences of working mathematicians in solving mathematical problems.

Having started teaching as an assistant professor of Computer Science at Carnegie Mellon in 1976, Lenat commenced work on his next AI program, known by the Greek word for "I discover" - Eurisko. Whereas the limitation with AM was that it was locked into following a fixed set of interestingness heuristics; Eurisko represented its heuristic rules as first class objects and hence it could explore, manipulate, and discover new heuristics just as it (and AM) explored, manipulated, and discovered new domain concepts.

Lenat returned to Stanford as an assistant professor of Computer Science in 1978 and continued his research building the Eurisko automated discovery and heuristic-discovery program. Lenat's paper "Heuretics: Theoretical and Experimental Study of Heuristic Rules" won the Best Paper award at the 1982 AAAI conference. Using Eurisko, Lenat also analyzed the rules of a complex role-playing game called Traveller Trillion Credit Squadron and entered and won a tournament dedicated to it two years running. 

In 1984 Lenat, in collaboration with John Seely Brown at Xerox PARC, published an analysis of the limitations of both his AM and Eurisko lines of research which concluded that progress toward real, general, symbolic AI would require a vast knowledge base of "common sense", suitably formalized and represented, and an inference engine capable of finding tens- or hundreds-deep conclusions and arguments that followed from the application of that knowledge base to specific questions and applications.

In the same year, having been appointed as a Principal Scientist at Microelectronics and Computer Consortium (MCC), Lenat commenced on the project to do just this. According to Wikipedia:

Cyc (pronounced SYKE) is a long-term artificial intelligence project that aims to assemble a comprehensive ontology and knowledge base that spans the basic concepts and rules about how the world works. Hoping to capture common sense knowledge, Cyc focuses on implicit knowledge that other AI platforms may take for granted. Cyc enables semantic reasoners to perform human-like reasoning and be less "brittle" when confronted with novel situations.

At the 400-person MCC, Lenat was able to have several dozen researchers work on that common sense knowledge base, rather than just a few graduate students.

In 1994 the project gained its own company, Cycorp, financed by various government organizations and private companies. Lenat was founder and CEO of Cycorp a position he still held at his death on August 31st, 2023. With his collaborators Lenat continued building his common sense engine, spending more than 2,000 human years on the project, writing more than 25 million rules. 

The only reference on I Programmer to Lenat and Cyc was in 2013 in an article about cutting edge research into image classifiers when I wrote:

The real progress in AI will only occur when the different areas come together to work on understanding and interacting with the world.

Aware of recent advances in other strands of AI research and the untrustworthy reputations of ChatGPT, Bard and co, Lenat proposed a new approach to future AI that would combine Cyc with large language models in an arXiv paper published in July 2023, co-authored with cognitive scientist Gary Marcus. 

The paper has the title Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc and starts from the premise that:

Generative AI, the most popular current approach to AI, consists of large language models (LLMs) that are trained to produce outputs that are plausible, but not necessarily correct. Although their abilities are often uncanny, they are lacking in aspects of reasoning, leading LLMs to be less than completely trustworthy. Furthermore, their results tend to be both unpredictable and uninterpretable.

The paper first itemises 16 desiderata for a trustworthy General AI, i.e capabilities including explanation, deduction, abductive reasoning and theory of mind and after commenting that LLMs struggle with most of them goes on to explain how Cyc handles them. Later it looks at opportunities for synergy between these two types of AI, providing some examples of where this has already happened.

It is worth quoting the ending paragraph of the paper:

In conclusion, there have been two very different types of AI’s being developed for literally generations, and each of them is advanced enough now to be applied -- and each is being applied -- on its own; but there are opportunities for the two types to work together, perhaps in conjunction with other advances in probabilistic reasoning and working with incomplete knowledge, moving us one step further toward a general AI which is worthy of our trust.

The New York Times obituary notice includes this comment from the paper's co-author Gary Marcus: 

"[Douglas Lenat] took on the project that no one else had the guts to take on. He never completely succeeded. But he showed us at least part of the way.” 

Douglas Bruce Lenat (1950–2023)

In this time when neural networks and statistical AI is yielding such startling results, it is easy to think of a huge project like Cyc as being a mistake. At the time of its creation this wasn't the prevailing view and Lenat was one of the first to express the belief that scaling was the problem we needed to overcome. He chose to scale logic and it was many years before we obtained any sucess from scaling networks. Back in the day there weren't many that thought that Cyc could be completely sucessful, but we all wanted to know how it might fail. 

More Information 

Douglas Lenat on Wikipedia

Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc by Doug Lenat, Gary Marcus

Related Articles

Runaway Success Of ChatGPT

Chat GPT 4 - Still Not Telling The Whole Truth

Never Ending Image Learner Sees, Understands, Learns

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info

<ASIN:0070155577>