Perceptron

Rosenblatt invented the single-neuron perceptron and a learning algorithm for it in 1958 and went on to prove some very powerful theorems about what it could learn. These theorems were a sort of guarantee that if something was learnable then the perceptron would learn it. The AI community at the time oversold the idea with demonstrations and outlandish claims for what could be done with one single perceptron.

Then the bubble burst. Minsky had met Seymour Papert and they were both thinking about the problem of working out exactly what a perceptron could do. The shocking truth that was revealed in the book that they wrote together in 1969 “Perceptrons” was that there really were some very simple things that a perceptron cannot learn. In particular concepts such as “odd” and “even” are beyond a perceptron, no matter how big it is or how long you give it to learn.

The perceptron book effectively discouraged any further work in the field simply because no funding organisation would give grants to crackpot AI research. For some 10 years, until the start of the 80s, the neural network approach to AI was effectively dead. A few places, mainly psychology labs and neurology labs still worked on the problem but progress was very slow. What started the revival was the discovery that multi-layer networks could be trained and they could solve the problems that Minsky and Papert had proved impossible for a single layer of perceptron.

Was the doubt that Minsky and Papert cast on neural networks justified?

At the time they were partly justified. Many people didn't really understand the result and interpreted it as proving that neural networks were a dead end. In fact the results only showed the single layer networks were a dead end. Of course at the time it was generally believed that there was no way to train a multi-layer network but in fact Paul Werbos published the algorithm that we now call back-propagation in 1974. It did take a very long time for the subject to recover from the misunderstanding of the book.

The Society of Mind

What is interesting here is that Minsky, after being so keen on the neural network approach, now seemed disillusioned with it.

Together with Papert he worked on a new theory “The Society of Mind”, a strange, almost mystical, theory that was explained in a book of the same name. The book, consisting of 270 interconnected one-page ideas, is certainly fun to read as the following excerpt demonstrates:

Holist: I'll prove no box can hold a mouse. A box is made by nailing six boards together. But it's obvious that no box can hold a mouse unless it has some "mouse-tightness" or "containment". Now, no single board contains any containment, since the mouse can just walk away from it. And if there is no containment in one board, there can't be any in six boards. So the box can have no mouse-tightness at all. Theoretically, then, the mouse can escape!

Citizen: Amazing. Then what does keep a mouse in a box?

Holist: Oh, simple. Even though it has no real mouse-tightness, a good box can "simulate" it so well that the mouse is fooled and can't figure out how to escape.

Logo and tutle graphics

At the same time he diversified into the psychology of education. “The Society of Mind” was based on information and theories of child development and this suggested ways to improve the early environment. Minsky and Papert invented the programming language, Logo, and Minsky built the first Logo Turtle.

Logo is derived from the AI language Lisp and the Turtle is based on the ideas of turtle geometry. Commands such as forward, turn right and so on can be used to make the Turtle draw geometric patterns. This approach allowed very young children to learn to program and develop concepts that involved projecting their own frame of reference onto the Turtle - in other words telling the turtle to turn to its left or right rather than their left or right.

In the long run, however, Logo has to be seen as something of an error – it is about as unsuitable as a first computer language as you can imagine. Turtles turned out to be more useful and enduring and as a result turn up in other languages and even as real robots.

Robotics

Despite having put a damper on the development of neural networks Minsky has continued to develop impressive robots and vision systems at MIT. In 1969 he built the first “human-like” robot hand manipulator, complete with tactile sensor. He thought up the idea for the “binary tree” robot manipulator – a many-armed robot in the form of a binary tree that to the casual observer seems far more difficult to program than a simple robot arm.

An early Minsky robot piling bricks.

He also had a long standing interest in music and why it exerts such a powerful influence over us. In 1970 he invented a musical variations synthesiser which composed music. His interests ranged into art, writing, dance and, in 1985, he became a founding member of the MIT Media Lab, where between 1988 and his death in 2016 he was the Toshiba Professor of Media Arts and Sciences.

<ASIN:0671657135>

<ASIN:0743276647>
<ASIN:1591025117>

<ASIN:0465046746>