Eliza and Parry

To give you some idea how effective this method is here is a conversation between two programs - Weizenbaum's Eliza, a therapist, and Parry, a program dating from 1972 written by psychiatrist Kenneth Colby which uses similar methods to pretend to be paranoid:

1 Parry - I don't understand your motives

2 Eliza - Why don't you understand my motives?

3 Parry - Shouldn't I be?

4 Eliza - Why do you say that just now?

5 Parry - That's the way it is.

6 Eliza - Suppose that's just the way it were not.

7 Parry - So?

8 Eliza - I am not sure I understand you fully.

9 Parry - You harp on the same questions too much.

10 Eliza - What does that similarity suggest to you?

11 Parry - I don't get you.

12 Eliza - Do you wish you did get I?

13 Parry - It's about time you showed some interest in my feelings.

14 Eliza - Oh? I showed some interest in your feelings?

and so on.

Remember there are two programs talking to each other here and not a human in sight.

At first it seems like a convincing interaction but the interaction is partly random responses and partly triggered by key words. For example, the opening sentence is one of a random selection from a paraniod's phrase book. The response is just Eliza using a simple rule to turn a question around.

The seemingly intelligent response at 10, where Elisa appears to understand the deeper significance of 9, is actually triggered simply by the presence of the word "same" in 9. The analysis of the conversation would carrry on to reveal that each response was triggered by a trivial feature of the previous input. And yet we are fooled because we read intelligent interaction into the conversation. Eliza and Parry borrow our skill of interpreting vague and ambiguous language to make themselves seem intelligent.

This is fun and people have claimed in the past that programs such as Eliza have passed the full open-ended Turing test because the judges who talked to them were not given any instructions to limit their conversation and yet still believed that they were talking to a human. Even after being told that they were talking to a program, many of the participants would not believe that it was so. Weizenbaum, the inventor of Eliza wrote:

" Eliza created the most remarkable illusion of having understood in the minds of many people who conversed with it... They would often demand to be permitted to converse with the system in private, and would, after conversing with it for a time, insist, in spite of my explanations, that the machine really understood them."

Restricted domains

What all this comes down to is that there is one restricted domain that AI programs can deal with and convince innocent people that they are intelligent.

The program that won the initial Loebner prize worked in the restricted domain "whimsical conversation" and of course this is exactly the domain that we have been discussing.

The winner of the initial Bronze Medal was Joseph Weintraub president of Thinking Software Inc with PC Therapist, a modified version of his commercially available program. It had taken more than three years to write and used an amount of memory - 5 MBytes - that was at the time considered to be "huge". It included and a knowledge base of thousands of amusing quotations and rules for conversation. It fooled five of the ten judges into thinking that it was a human. You can see a sample of the dialog from this program below.

The winning dialog

TURING TEST TRANSCRIPT FOR TERMINAL 5

JUDGE 7 - It's great to finally get started. Do you know what the hold-up was?

TERMINAL 5 - True art selects and paraphrases, but seldom gives a verbatim translation. We were discussing you, not me. How ya doin', judge?

JUDGE 7 - Greetings and Salutations.

TERMINAL 5 - I care about you... I can help you, my dear judge Do you prefer wine, women or song? How does that make you feel?

JUDGE 7 - All of the above

TERMINAL 5 - It is as absurd to say that a man can't love one woman all the time as it is to say that a violinist needs several violins.

JUDGE 7- Wonderful.

TERMINAL 5 - I just got wonderful news from my real estate agent in Florida - they found land on my property.

JUDGE 7 - Wonderful

TERMINAL 5 - I wonder what it means to be human. What was I saying...? Please go on.

JUDGE 7 - Wonderful

TERMINAL 5 - Please don't repeat yourself Wouldn't you agree? What does that suggest to you?

JUDGE 7 - Interesting.

TERMINAL 5 - I prefer an interesting vice to a virtue that bores. Do you watch much tv? You seem to have avoided speaking of your friends altogether.

JUDGE 7 - Fail to comprehend

TERMINAL 5 - It is infinitely better to take the side of the wicked who prosper. Do you feel strongly about discussing such things? My god please help me!

JUDGE 7 - Go for it

TERMINAL 5 - You're cute... would you like to fool around a little? How important is that to you? Can you believe this crazy competition?

.....

TERMINAL 5 - I have you under complete hypnotic control... you are more relaxed than ever before, my dear judge.

JUDGE 7- go

TERMINAL 5 - Are you a computer illiterate? one of those people frightened of computers? Do you trust me? What does that suggest to you?

JUDGE 7 - fantastic

TERMINAL 5 - Intelligence appers to be the thing that enables a man to get along without an education.

JUDGE 7 - into

JUDGE 7 -

TERMINAL 5 - You must have more to say than that? Whimsy can be flimsy, right? Do you feel strongly about discussing such things?

CHANGE OF JUDGES

JUDGE 10 - Hello! You seem to be feeling quite witty today.

TERMINAL 5 - Hello, judge, what is your mood today? happy? sad? ready for some fun? Hello - how's the weather up there today? Lets talk about psychology!

JUDGE 10 - It has been a long day! Now I think I'm ready for some fun. The weather isn't very pleasant today. Do you have any jokes that could lift the clouds?

TERMINAL 5 - Everybody talks about the the weather but nobody seems to do much about it. Why do you need to tell me now you think you're ready for some fun? Groannnnn..............

JUDGE 10 - Well you did ask me!

TERMINAL 5- Now you really have outdone yourself in asking stupd things!. Woof....woof....woof....Why do you think I did ask you?

JUDGE 10 - Do you find insults to be humorous? I think maybe you asked me because you don't really know me and are trying to break the ice.

TERMINAL 5 - Is that the real reason? Was that whimsical enough for you? Oh, I find insults to be humorous?

JUDGE 10 - I guess that sometimes I have quite a sarcastic humor, but I generally don't insult people that I have just met!

TERMINAL 5 - You will never find time for anything - if you want time, you must make it. Ipso facto....... Can you elaborate on that?

JUDGE 10 - You're trying to make this difficult for me aren't you? I just explained that I was ready for fun!

TERMINAL 5 - Would you prefer it if it was not true that you're trying to make this difficult for you aren't you? I just explained that I was ready for fun! I am,therfore I compute.... What makes you think I am trying to make this difficult for you aren't I?

JUDGE 10 - That last passage is a little confusing. Now you're really trying to let me have it, right? I guess that it is my fault. How about a joke now.

TERMINAL 5 - How would you solve that? Ipso facto nonsensico...... Does it please you to believe I am really trying to let you have it?

JUDGE 10 - I don't know how I would solve it, but it is sort of fun kidding around with you!

TERMINAL 5 - Is ditchwater dull? naturalists with microscopes have told me that it teems with quiet fun.

CHANGE OF JUDGES

Joseph Weizenbaum, the man who invented Eliza, commented that he way struck by how similar PC Therapist was to the program that he had written and also said that such programs were difficult to trick because it was difficult for humans to assess the nonsensical replies from the program. Well I would have to agree with him on both counts and I am sure that he must be amused at how well his invention has fared over the years.

I predicted at the time that unless the rules were changed PC Therapist or something from this class of programs was likely to win every year without making any great breakthroughs towards the type of AI required to pass the "full" Turing test. In fact Wietraub and PC Therapist won the following two contests and that of 1995 and, according to Wikipedia the Loebner Prize is now considered to be for "the chatterbot considered by the judges to be the most human-like."

The Bronze Medal has been awarded on an annual basis and the only significanet change from the initial competition is that since 1995 the discussion has been unrestricted so making the challenge rather harder. Transcripts from recent competitions can be found on the Loebner Prize website. A new category was also introduced - a Silver Medal plus the sum of  $25,000 for the first chatterbot that judges cannot distinguish from a real human and that can convince judges that the human is the computer program. This one time prize has yet to be won but does not seem an impossible challenge. However, the Gold Medal reserved for the first chatterbot that judges cannot distinguish from a real human in a Turing test that includes deciphering and understanding text, visual, and auditory input still seems a distant goal. Once this is achieved, the annual competition will end.

Latest news:

The 2010 prize was awarded on October 23rd. See Meet Suzette, prize winning chatbot for a report of the event and a link to follow to chat to the winner, Suzette.

<ASIN:1852336722>

<ASIN:0262631873>