{"id":766,"date":"2019-01-01T12:20:34","date_gmt":"2019-01-01T12:20:34","guid":{"rendered":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/?p=766"},"modified":"2019-01-01T12:20:34","modified_gmt":"2019-01-01T12:20:34","slug":"the-turing-test-is-not-a-gallup-poll-and-it-was-not-passed","status":"publish","type":"post","link":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/2019\/01\/01\/the-turing-test-is-not-a-gallup-poll-and-it-was-not-passed\/","title":{"rendered":"The Turing Test Is Not A Gallup Poll (And It Was Not Passed)"},"content":{"rendered":"

<\/a>It has been reported that the Turing Test has been passed, 64 years after it was first proposed by Turing, because 32% of judges mistook a computer programme for a real 13-year old Ukrainian boy called Eugene Goostman in a 5-minute test<\/a>. <\/p>\n

Nothing of the sort. Really passing the Turing Test would require designing a system that has real, lifelong verbal capacity, indistinguishable from a real pen-pal, not just fooling an arbitrary percentage of interrogators in a series of 5-minute exchanges!<\/p>\n

At the beginning of his 1950 paper Turing had written:<\/p>\n

Turing:<\/b> \u201c[A] statistical survey such as a Gallup poll [would be] absurd [as a way to define or determine whether a machine can think]\u201d <\/i>(Turing 1950)<\/p><\/blockquote>\n

Taking a statistical survey like a Gallup Poll \u2014 to find out people’s opinions of what thinking is \u2014 would indeed be a waste of time, as Turing points out. Later in the paper, however, in a throwaway remark that is merely his personal prediction about progress in attempts to pass his Test, he mentions the equivalent of a statistical survey in which 30% of interrogators will be successfully fooled for five minutes: <\/p>\n

Turing:<\/b> “I believe that in about fifty years’ time it will be possible, to programme computers… [to] play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning<\/i>.\u201d (Turing 1950)<\/p><\/blockquote>\n

No doubt this party-game\/Gallup-Poll criterion can be met by today’s computer programs — but that remains as meaningless a demographic fact today as it was when predicted 64 years ago: Like any other science, cognitive science is not the art of fooling some or most of the people for some or most of the time! The candidate must really have the generic performance capacity of a real human being<\/i> — capacity that is totally indistinguishable from<\/i> that of a real human being to<\/i> any real human being (for a lifetime, if need be!). No tricks: real performance capacity (Harnad 2008).<\/p>\n

Turing was not only the co-inventor of the computer and the code-breaker of the Nazis\u2019 Enigma Machine, thereby helping the Allies win World War II, but with what came to be called the \u201cTuring Test\u201d he also set the agenda for what would eventually come to be called \u201ccognitive science\u201d: the science of explaining how the mind works.<\/p>\n

Turing\u2019s idea was simple: Stop worrying about what the mind \u201cis\u201d and explain instead what the mind does. If you can design a system that can do everything that a person with a mind can do \u2013 and can do it so that people cannot tell it apart from a real person \u2013 then that system will have passed the Turing Test, and the explanation of how that system works will be the explanation of how the mind works.<\/p>\n

But the lion\u2019s share of the enormous research agenda proposed by Turing for cognitive science is getting the system to be able to do everything a person with a mind can do. Testing whether people can tell the candidate apart from a real person only becomes relevant at the endgame, once the system already has our generic performance capacities. And we are nowhere near having designed a system that can do everything a person with a mind can do. Not even if we restrict the test to everything a mind can do verbally. (The real Turing Test will of course have to be robotic, not just verbal, because what we can do is not just what we can do with our mouths! But lets set aside for another discussion the \u201csymbol grounding problem<\/a>\u201d of whether computation alone can indeed do everything the mind can do.)<\/p>\n

It should be obvious from all this that the Turing Test is not \u2013 and never was \u2013 about fooling anyone, let alone fooling some people, some of the time. It is about designing \u2013 indeed \u201creverse-engineering\u201d — a system that is really able to do anything an ordinary person can do, any time, as long as you like, indistinguishably from the way a real person does it. Nothing about 5-minute tests and percentages of judges that think the candidate is or isn\u2019t a real person (although obviously eventual success can only be achieved by degrees).<\/p>\n

The Turing Test has captured the imagination of the general public partly because of our interactions with computers that are able to do more and more things that only people with minds had been able to do. Another reason has been the growth in the number of science fiction books and movies about computers and robots that have \u2013 or seem to have \u2013 minds. But the biggest reason for the fascination is the \u201cother minds problem\u201d itself \u2013 the very problem that the Turing Test is meant to resolve:<\/p>\n

We are not mind-readers. The only one I can know has a mind is myself; we\u2019ve known that since at least Descartes\u2019 famous \u201cI think therefore I am.\u201d For all bodies other than my own, the only way I can infer whether they indeed have a mind is if they can do what minds can do<\/i>. I can\u2019t observe other minds, but I can observe what they can do. So Turing\u2019s real insight was that Turing-testing is — and always has been — our only means of mind-reading. Hence once we have designed a system that can do anything a person with a mind can do, indistinguishably from a person with a mind, not only will we be in no better or worse a position to know whether that system really has a mind than with any other person, but we will come as close as it is possible to come to having explained how the mind works.<\/p>\n

But that\u2019s certainly not where we are when we have a system that can fool 30% of people for 5 minutes. And Turing certainly never said, implied or intended any such thing.<\/p>\n

Harnad, S. (1992) The Turing Test Is Not A Trick: Turing Indistinguishability Is A Scientific Criterion<\/a>. SIGART Bulletin<\/i> 3(4) (October 1992) pp. 9 – 10. <\/p>\n

Harnad, S. (2008) The Annotation Game: On Turing (1950) on Computing, Machinery and Intelligence<\/a>. In: Epstein, Robert & Peters, Grace (Eds.) Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer.<\/i> Springer <\/p>\n

Turing, A. M. (1950). Computing machinery and intelligence<\/a>. Mind<\/i>, 433-460.<\/p>\n","protected":false},"excerpt":{"rendered":"

It has been reported that the Turing Test has been passed, 64 years after it was first proposed by Turing, because 32% of judges mistook a computer programme for a real 13-year old Ukrainian boy called Eugene Goostman in a 5-minute test. Nothing of the sort. Really passing the Turing Test would require designing a … <\/p>\n

Continue reading “The Turing Test Is Not A Gallup Poll (And It Was Not Passed)”<\/span><\/a><\/p>\n","protected":false},"author":3074,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[1],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts\/766"}],"collection":[{"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/users\/3074"}],"replies":[{"embeddable":true,"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/comments?post=766"}],"version-history":[{"count":1,"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts\/766\/revisions"}],"predecessor-version":[{"id":767,"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts\/766\/revisions\/767"}],"wp:attachment":[{"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/media?parent=766"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/categories?post=766"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/tags?post=766"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}