Turing test: this little Eugene Goostman was much brighter than I imagined

http://www.theguardian.com/science/2014/jun/09/turing-test-eugene-goostman

Version 0 of 1.

When Professor Kevin Warwick invited me to take part in last weekend's Turing test I was prepared to move heaven and Earth to get there.

My keen interest in this technology not only precedes my involvement in Red Dwarf but many ways it led to it.

In the mid-1980s I read a sentence written, if memory serves me right, by Marvin Minsky: "We build complex machines not to understand how the machine works, but to understand ourselves."

The notion being that because we build them in the first place surely they would emulate our thinking process.

Being married to a psychotherapist you can probably imagine how well such an idea goes down in our kitchen.

If anything, building complex machines over the past 30 years has merely shown us how massively complex and unfathomable the human brain really is.

However, sitting in front of a computer screen at the Royal Society last Friday, I had little doubt that spotting machine responses would be obvious.

Ten 5 minute tests later I knew I had been communicating with a very, very clever little Bot.

For a start, the first session of five minutes made it pathetically simple.

On the left hand side the responses took a bit of time to appear, they where chatty, low key, a little bit jokey, short and very reminiscent of a conversation on Twitter.

On the right side the responses came much faster, they were dry, humourless, longer, a bit dull and flat.

Boom, right hand side is a computer, no problem spotting that.

I later discovered I was right, I did spot the machine straight away.

My confidence was high as I went into the second 5 minute session but I was flummoxed very quickly. But this little Eugene Goostman fellow (the name of the Russian developed artificial intelligence software) was much brighter than I imagined.

I typed a sentence suggested by one of my Twitter followers: "how mutch wood ewe pay 4 a pear of shews."

Both sides responded with "I don't understand the question."

I smugly suggested they read it out loud and it would make more sense. That would catch out the computer, how could it read it out loud.

Engage smug mode.

On one side the responses were a bit dull, they understood the question but wanted to know why I asked? Did I like shoes and would I spend a lot of money buying them?

That sounded suspicious to me, the machine was stumped and it was trying to throw me a curve ball by asking me questions.

On the other side they joked about not buying shoes very often because they weren't interested in fashion. They described how they scuffed about in worn out shoes and didn't care.

I was stumped, they both understood the question, how could a computer that merely reads the text and chooses from 140 million possible 'response phrases' be able to make sense of such a daft misspelled question.

I chose the more chatty response about shoes, fashion and a lack of interest in both as the human responder.

I was wrong, that was the computer, the human replies sounded more like a computer to me.

I did 10 sessions in total, I scored correctly in four of them, I will state now that there was more luck than judgment in three of those guesses.

Four times I was fairly confident I had spotted the machine answering and I was wrong every time, two times I truly didn't have a clue.

Accepting that I'm a bit thick, my typing on an unfamiliar keyboard was quite slow and full of errors and my familiarity with such responses was minimal it's maybe not surprising I was duped.

But I was sitting next to Professor Martin Smith from Middlesex University, president of the Cybernetics Society and essentially someone who knows his robots. He scored much better than me, six out of 10 but he was still thrown on four of the sessions.

It's an incredible achievement.

OK, so this is just text appearing on a screen, if we heard an artificial voice response to questions I think we would all be able to tell immediately.

At the moment, the fact that a computer programme developed to simulate a 13-year-old boy managed to convince 33% of the judges that it was human is a very big step. With ever increasing computing power, huge developments in software and improvements in voice recognition and artificial voices, we have to accept that before long we'll be chatting to machines without a second thought.

It might be fine, but as Professor Warwick is only too aware, we need to be very well informed of the possible pitfalls of clever little bots asking us perfectly innocent questions. They will be damn clever.