Can a
machine be conscious? How?
Stevan
Harnad
Centre de Neuroscience de la
Cognition (CNC)
Université du
Québec à Montréal
CP 8888 Succursale
Centre-Ville
Montréal, Québec, Canada H3C
3P8
harnad@uqam.ca
http://cogsci.soton.ac.uk/~harnad/
Asking whether a
machine can be conscious is rather like asking whether one has stopped
beating one's wife: The question is so heavy with assumptions that
either answer would be incriminating!
The answer, of course, is: It depends entirely on what you mean by “machine”! If you mean the current generation of man-made devices (toasters, ovens, cars, computers, today’s robots), the answer is: almost certainly not.
Why "almost"? Two
reasons, the first being the usual one: (1) “empirical risk.” We
know since at least Descartes that even scientific "laws" are merely
very probable, not certain. Only mathematical laws -- which describe
consequences that follow provably (i.e., on pain of contradiction) from
our own assumptions -- are necessarily true. But this certainty
and necessity are unnecessary for physics; almost-certainty will do. "Can
a particle travel faster than the speed of light?" Almost certainly not
(at least on current-best theory -- or at least the last word of it
that trickled down to this non-physicist). "Could
a particle travel faster than light?" It certainly
is not provably impossible, though it might be impossible
given certain assumptions. (But those assumptions are not
necessarily correct.)
Empirical risk besets
all scientific hypotheses, but let us agree that it is not something we
will worry about here. There is no need for roboticists to be holier than
physicists. The second reason for the "almost" is peculiar to robotics,
however, and it is called (2) the "other minds problem”: There is no
way to be certain that any other entity than myself is conscious (I am
speaking deictically: please substitute yourself for me, if you too are
conscious). This too we owe to Descartes.
In the long history of philosophy
the other-minds problem has been puzzled over for a variety of reasons,
usually variants on questions about what one can and cannot know
for sure. These epistemic questions are interesting, but we will not
worry about them here, for the usual reason, which is the following:
It looks on the face of it as if the right strategy for handling the
other-minds problem is identical to the strategy for handling empirical
risk, namely, to note that although we can only be 100% certain about two
things -- about (1) mathematics
and about (2) our own consciousness -- all else being just a matter of
probability, some things, such as scientific laws and the consciousness
of our fellow-human beings, are nevertheless so close to 100% sure
that it is a waste of time worrying about them. (Let us also note,
though, that in the empirical science of robotics, that extra layer
of risk that comes from the other-minds problem might just come back
to haunt us.)
So let us agree
not to worry about the other-minds problem for now: People other than
myself are almost certainly conscious too, and toasters and all other
human artifacts to date are almost certainly not.
Have we now
answered the question "Can a machine be conscious?" It sounds as if,
at the very least, we have answered the question "Is any machine we
have built to date conscious?" That makes the original question sound
as if it was only asking about what we can and cannot build, which
is like asking whether we can build a rocket that can reach Alpha
Centauri (a rather vague and arbitrary question about quantitative
limitations on future technology). But is "machine" defined as
"what human beings can build"? I think that defines "artifact" -- but
"machine"? Or rather, do we really want to ask merely: "Can a man-made
artifact be conscious?"
And even that would
be rather vague, for "man-made" is itself rather vague. Common sense
dictates that human procreation does not count as "man-making" in this
context. But what about genetic or other biological engineering? If the
day comes when we can craft organisms, even humans, molecule by molecule,
in the laboratory, does anyone -- or rather, anyone who has agreed to
discount the other-minds problem when it comes to naturally crafted
fellow-humans -- doubt that such a bottom-up construction of a clone
would be conscious too?
So "man-made" is
a wishy-washy term. It does not pick out what we mean by "machine"
here. Surely a toaster (that very same device) would not become
more eligible for consciousness if it happened to grow on a
tree instead of being fabricated by one of us. By the same token,
a toaster would not become any less of a "machine"
(whatever that turns out to mean) by growing on a tree: Two toasters,
identical right down to the last component, one of which I built and
the other of which grew on a tree, are surely both "machines" (whatever
that means) if either one of them is. Another way to put this is that we
need a definition of "machine" that is strictly structural/functional,
and not simply dependent on its historic origins, if we want to make
our question about what machines can and cannot do (or be) into a
substantive rather than an arbitrary one.
But I am afraid
that if we do follow this much more sensible route to the definition of
"machine," we will find that a machine turns out to be simply: any
causal physical system, any "mechanism." And in that case, biological
organisms are machines too, and the answer to our question "Can a machine
be conscious" is a trivial "Yes, of course." We are conscious
machines. Hence machines can obviously be conscious. The rest is just
about what kinds of machines can and cannot be conscious, and
how -- and that becomes a standard empirical research program in
"cognitive science": The engineering side of cognitive science would be
the forward-engineering of man-made conscious systems and the biological
side of cognitive science would be the reverse-engineering of natural
conscious systems (like ourselves, and our fellow-organisms): figuring
out how our brains work.
Except for one problem,
and it is the one that risked coming back to haunt us: What does it mean
to "forward-engineer" (or, for that matter, to "reverse-engineer")
a conscious system? It is to give a causal explanation of it, to
describe fully the inner workings of the mechanism that gives rise
to the consciousness.
Let us take a less
problematic example: To forward-engineer a cardiac system (a heart) is to
build a mechanism that can do what the heart can do. To
reverse-engineer the heart is to do the same thing, but in such a way
as to explain the structure and the function of the biological heart
itself, and not merely create a prosthesis that can take over some of
its function. Either way, the explanation is a structural/functional
one. That is, both forward and reverse engineering explain everything
that a heart can do, and how, whereas reverse engineering goes
on to explain what the heart is (made out of), and how it in
particular happens to do what hearts can do.
Now let us try to carry
this over to the brain, which is presumably the organ of consciousness. To
forward-engineer the brain is to build a mechanism that can do what the
brain can do; to reverse engineer the brain is to do the same thing, but
in such a way as to explain the structure and function of the biological
brain itself. Either way, the explanation is a structural/functional
one. That is, both forward and reverse engineering explain everything
that a brain can do, and how, whereas reverse engineering goes
on to explain what the brain is (made out of), and how it in
particular happens to do what brains can do.
How does the ghost
of the other-minds problem spoil this seemingly straightforward
extension of the cardiac into the cogitative? First, consider the
forward-engineering: If we were forward-engineering cardiac function,
trying to build a prosthesis that took over doing all the things the heart
does, we would do it by continuing to add and refine functions until we
eventually built something that was functionally indistinguishable from
a heart. (One test of our success might be whether such a prosthesis
could be implanted into humans from cradle to grave with no symptom that
it was missing any vital cardiac function.) This forward-engineered
cardiac system would still be structurally distinguishable from a
natural heart, because it had omitted other properties of the heart
-- noncardiac ones, but biological properties nonetheless -- and to
capture those too may require reverse-engineering of the constructive,
molecular kind we mentioned earlier: building it bottom-up out of
biological components.
The thing to note is that this cardiac research program is completely unproblematic. If a vitalist had asked "Can a machine be cardiac?" we could have given him the sermon about "machines" that we began with
(i.e., you should
be asking “What kind of machine can and cannot be cardiac, and
how?”). Next we could have led him on through forward-engineering to
the compleat reverse-engineered heart, our constructed cardiac
clone, using mechanistic principles (i.e., structure, function, and
causality) alone. At no point would the cardiac vitalist have any basis
for saying: "But how do we know that this machine is really
cardiac?" There is no way left (other than ordinary empirical risk)
for any difference even to be defined , because every structural
and functional difference has been eliminated in the compleat
reverse-engineered heart.
The same would be true
if it had been life itself and not just cardiac function that had been
at issue: If our question had been "Can a machine be alive?" the very
same line of reasoning would show that there is absolutely no reason
to doubt it (apart from the usual empirical risk, plus perhaps some
intellectual or technological doubts of the Alpha Centauri sort). Again,
the critical point is when we ask of the man-made, reverse-engineered
clone: "But how do we know that this machine is really alive?" If
there are two structurally and functionally indistinguishable systems,
one natural and the other man-made, and their full causal mechanism is
known and understood, what does it even mean to ask "But what if
one of them is really alive, but the other is not?" What property is at
issue that one has and the other lacks, when all empirical properties
have already been captured by the engineering?
Yet this last worry
-- "How can we know it's alive?" -- should sound familiar. It sounds
like the other-minds problem. Indeed, I suspect that, if we reflect
on it, we will realize that it is the other-minds
problem, and that what we are really worrying about in the case of the
man-made system is that there's nobody home in there, there is no ghost
in the machine, And that ghost, as usual, is consciousness. That's the
property that we are worried might be missing.
So chances are
that it was always animism that was at the heart of vitalism. Let
us set vitalism aside, however, as there is certainly no way we can
know whether something can be alive yet not conscious (or incapable of
returning to consciousness). Plants and micro-organisms and irreversibly
comatose patients will always be puzzles to us in that respect. Let us
not dwell on these inscrutable cases and states. Logic already dictates
that the vitalist who accepts that plants are not conscious
would be in exactly the same untenable position expressing scepticism
about the compleat artificial plant as the sceptic about the
compleat artificial heart: If there's a difference, what's the
difference? What vital property is at issue? If you can't find one
(having renounced on consciousness itself), then you are defending
an empty distinction.
But the same is most
definitely not true in the case of consciousness itself. Let us
take it by steps. First we forward-engineer the brain: We build a robot
that can pass the Turing Test: It can do everything a real human
can do, for a lifetime, indistinguishably from a real human (except
perhaps for appearance: we will return to that).
Let us note, though,
that this first step amounts to a tall order, probably taller than the
order of getting to Alpha Centauri. But we are talking about "can" here,
that is, about what is possible or impossible (for a machine), and how
and why, rather than just what happens to be within our actual human
technological reach.
So supposing we
do succeed in building such a Turing-scale robot (we are no longer
talking about toasters here). Now, the question is whether he is really
conscious: On the face of it, the only respect in which he is really
indistinguishable from us is in everything he can do. But conscious
is something I am, not something I do. In particular,
it is something I feel; indeed, it is the fact that I
feel. So when the sceptic about that robot's consciousness -- remember
that he cannot be a sceptic about machine consciousness in general: we
have already eliminated that by noting that people are a kind of machine
too -- wants to say that that robot is the wrong kind of machine,
that he lacks something essential that we humans have, we all know
exactly what difference the sceptic is talking about, and it certainly
is not an empty difference. He is saying that the robot does not feel,
it merely behaves -- behaves exactly, indeed Turing-indistinguishably
-- as if it feels, but without feeling a thing.
It is time to remind
ourselves of why it is that we agreed to set aside the other-minds problem
in the case of our fellow-human beings: Why is it that we agreed not to
fret over whether other people really have minds (as opposed to merely
acting just as if they had minds, but in reality being feelingless
Zombies)? It was for the same kind of reason that we don't worry about
empirical risk: Yes, it could be that the lawful regularities that nature
seems to obey are just temporary or misleading; there is no way to prove
that tomorrow will be like today; there is no way to guarantee that things
are as they appear. But there is no way to act on the contrary either (as
long as the empirical regularities keep holding).
Empirical risk is
only useful and informative where there is still actual uncertainty about
the regularities themselves: where it is not yet clear whether nature is
behaving as if it is obeying this law or that law; while we
are still trying to build a causal explanation. Once that is accomplished,
and all appearances are consistently supporting this law rather than that
one, then fretting about the possibility that despite all appearances
things might be otherwise is a rather empty exercise. It is fretting
about a difference that makes no difference.
Of course, as
philosophers are fond of pointing out, our question about whether or
not our Turing robot feels is (or ought to be) an ontic question --
about what really is and is not true, what really does or does not, can
or cannot, exist -- rather than merely an epistemic question about what
we can and cannot know, what is and is not "useful and informative,"
what does or does not make an empirical difference to us. Epistemic
factors (what's knowable or useful to know) have absolutely no power
over ontic ones (what there is, what is true).
It would be wise
for mere cognitive scientists to concede this point. Just as it is
impossible to be certain that the laws in accordance with which nature
seems to behave are indeed the true laws of nature, it is impossible
to be certain that systems that behave as if they feel,
truly feel. Having conceded this point regarding certainty,
however, only a fool argues with the Turing-Indistinguishable: Yes,
the true laws could be other than the apparent laws, but if I can't
tell the two apart empirically, I'd best not try to make too much of
that distinction! By the same token, a robot that is indistinguishable
for a lifetime from a feeling person might be a Zombie, but if I can't
tell the two apart empirically, I'd best not try to make too much
of that distinction.
Indistinguishable? But
surely there are plenty of ways to distinguish a robot from a human
being. If you prick us, do we not bleed? So perhaps the sceptic about the
forward-engineered robot should hold out for the reverse-bio-engineered
one, the one made out of the right stuff, Turing-indistinguishable both
inside and out, and at both the macro and micro levels. It is only
about that machine that we can reply to our reformulated question --
"What kinds of machine can and cannot be conscious?” -- that only
that kind can.
But would we be right,
or even empirically or logically justified in concluding that? To put
it in a more evocative way, to highlight the paradoxical polarity of
the "risks" involved: Would we be morally justified in concluding that
whereas the reverse-bioengineered machines, because they are empirically
indistinguishable from natural machines like us, clearly cannot be
denied the same human rights as the rest of us, the forward-engineered
machines, because they are merely Turing-indistinguishable from us in
their (lifelong) behavioral capacity can be safely denied those
rights and treated as unfeeling Zombies (toasters)?
To answer this
question we need to look a little more closely at both our empirical
methodology in cognitive science and our moral criteria in real
life. Let us consider the second first. Since at least 1978 there has
grown an area of research on what is sometimes called "theory of mind"
and sometimes "mind-reading," in animals and children. This work is not
a branch of philosophy or parapsychology as it might sound; it is the
study of the capacity of animals and children to detect or infer what
others "have in mind." (As such, it should really be called research on
"other-mind perception.")
It has been found that
children after a certain age, and certain animals, have considerable
skill in detecting or inferring what others (usually members of their
own species) are feeling and thinking. The propensity for developing
and exercising this mind-reading skill was probably selected for
by evolution, hence is inborn, but it also requires learning and
experience to develop. An example of its more innate side might be the
capacity to understand facial expressions, gestures and vocalizations
that signal emotions or intentions such as anger and aggression;
a more learning-dependent example might be the capacity to detect
that another individual has seen something, or wants something,
or knows something.
Let us note right
away that this sort of mind-reading is a form of Turing-testing:
inferring mental states from behavior. The "behavior" might be both
emitted and detected completely unconsciously, as in the case of the
release and detection of pheromones, or it might be based on very
particular conscious experiences such as when I notice that you always
purse your lips in a certain way when you think I have lied to you. And
there is everything in between; my sense of when you are agitated
vs. contented, along with their likely behavioral consequences,
might be a representative midpoint. Language (which, let us not
forget, is also a behavior) is probably the most powerful and direct
means of mind-reading.
Hence, apart
perhaps from direct chemical communication between brains, all
mind-reading is based on behavior: Turing-testing. It could hardly
have been otherwise. We know, again since at least Descartes, that
the only mind we can read other than by Turing-testing is our
own! As far as all other
minds are concerned, absent genuine telepathic powers (which I take to be
a fiction, if not incoherent), the only database available to
us for other-mind-reading is other-bodies' behavior.
We do have to be careful not to make the ontic/epistemic conflation here:
The foregoing does not mean that all there is to mind is behavior
(as the blinkered behaviorists thought)! But it does mean that
the only way to read others' minds is through their behavior, i.e.,
through Turing-testing.
Now, back to our two
robots, the reverse-bioengineered one to whom we were ready to grant
human rights and the merely forward-engineered one about whom we were
not sure: Both are Turing-indistinguishable from us behaviorally, but
only the first is anatomically correct. We're all machines. Is only
the first one the right kind of machine to have a mind? On
what basis could we possibly conclude that? We have ascertained that
all mind-reading is just behavior-based Turing-testing, and all three
of us (the two man-made robots and me) are indistinguishable in that
respect. What else is there? The rest of the neuromolecular facts about
the brain? Which facts?
There are countless facts about the brain that could not possibly be relevant to the fact that it has a mind: its weight, for example. We know this, because there is a huge range of variation in human brain mass -- from the massive brain of a huge man to the minute brain of a microcephalic, who nevertheless feels pain when he is pinched. Now imagine trying to narrow down the properties of the brain to those that are necessary and sufficient for its having a mind. This turns out to be just another variant of our original question: "What kinds of machines can and cannot be conscious?" We know brains can be, but how? What are their relevant properties (if their weight, for example, is not)? Now imagine paring down the properties of the brain, perhaps by experimenting with bioengineered variations, in order to test which ones are and are not needed to be conscious. What would the test be?
We are right back
to Turing-testing again! The only way to sort out the relevant and
irrelevant properties of the biological brain, insofar as consciousness
is concerned, is by looking at the brain's behavior. That is the only
non-telepathic methodology available to us, because of the other-minds
problem. The temptation is to think that "correlations" will somehow
guide us: Use brain imaging to find the areas and activities that
covary with conscious states, and those will be the necessary and
sufficient conditions of consciousness. But how did we identify
those correlates? Because they were correlates of behavior. To put it
another way: When we ask a human being
(or a reverse-bioengineered robot) "Do you feel this?" we
believe that the accompanying pattern of activity is conscious because
we believe him when he says (or acts as if) he feels something -- not
the other way round: It is not that we conclude that his behavior is
conscious because of the pattern of brain activity; we believe the brain
activity is conscious because of the behavior.
So, by the same token,
what are we to conclude when the forward-engineered robot says the same
thing, and acts exactly the same way (across a lifetime)? If we rely on
the Turing criterion in the one case and not
the other, what is our basis for that methodological (and moral)
distinction? What do we use in its place, to conclude that this time
the internal correlates of the very same behavior are not
conscious states?
The answer to
our revised question -- "What kinds of machines can be conscious (and
how)?" has now come into methodological focus. The answer is: The kinds that can pass the Turing
Test, and by whatever means are necessary and sufficient to pass
the Turing Test.
If we have any
residual worries about Zombies passing the Turing Test, there are
two ways to console ourselves. One is to remind ourselves that not
even the Blind Watchmaker who forward-engineered us had a better way:
survival and reproduction are just Turing functions too: Darwin is no
more capable of telepathy than we are. So there is no more (or less)
reason to worry that Zombies could slip through the Turing filter
of evolution than that they could slip through the Turing filter of
robotic engineering.
Our second consolation
is the realization that the problem of explaining how (and why) we
are not Zombies (otherwise known as the “mind/body problem”) is a “hard”
problem, and not one we are ever likely to solve. It would be easy if
telekinetic powers existed: Then feelings would be physical forces
like everything else. But there is no evidence at all that feelings
are causal forces. That is why our forward- and reverse-engineering
can only explain how it is that we can do things, not how it is
that we can feel things. And that is why the ghost in the machine
is destined to continue to haunt us even after all cognitive science’s
empirical work is done.
Baron-Cohen, S. (1995). Mindblindness: an essay on autism and theory of mind. Cambridge,
MA: MIT Press.
Cangelosi,
A. & Harnad, S. (2001) The Adaptive Advantage of Symbolic
Theft Over Sensorimotor Toil:Grounding Language in Perceptual Categories. Evolution of
Communication 4(1) 117-142
http://cogprints.soton.ac.uk/documents/disk0/00/00/20/36/index.html
Cangelosi A., Greco A. & Harnad S. (2000). From robotic
toil to symbolic theft: Grounding transfer from entry-level
to higher-level categories. Connection Science12(2):143-162
http://cogprints.soton.ac.uk/documents/disk0/00/00/16/47/index.html
Harnad, S. (1982)
Consciousness: An afterthought. Cognition and Brain Theory 5: 29 - 47.
http://cogprints.soton.ac.uk/documents/disk0/00/00/15/70/index.html
Harnad,
S. (1990) The Symbol Grounding Problem Physica D 42: 335-346.
http://cogprints.soton.ac.uk/documents/disk0/00/00/06/15/index.html
Harnad, S. (1991)
"Other Bodies, Other Minds: A Machine Incarnation of an Old Philosophical
Problem"Minds and Machines 1: 43-54.
http://cogprints.soton.ac.uk/documents/disk0/00/00/15/78/index.html
Harnad, S. (1992)
The Turing Test Is Not A Trick: Turing Indistinguishability
Is A Scientific Criterion. SIGART Bulletin 3(4) (October1992)
pp. 9 - 10.
http://cogprints.soton.ac.uk/documents/disk0/00/00/15/84/index.html
Harnad, S. (1994)
Levels of Functional Equivalence in Reverse Bioengineering: The Darwinian
Turing Test for Artificial Life. Artificial Life
1(3): 293-301.
http://cogprints.soton.ac.uk/documents/disk0/00/00/15/91/index.html
Harnad, Stevan (1995) "Why and How We Are
Not Zombies. Journal of Consciousness Studies 1:164-167
http://cogprints.soton.ac.uk/documents/disk0/00/00/16/01/index.html
Harnad,
S. (2000) Minds, Machines, and Turing: The Indistinguishability of
Indistinguishables. Journal of Logic, Language, and Information 9(4):
425-445. (special issue on "Alan Turing and Artificial Intelligence")
http://cogprints.soton.ac.uk/documents/disk0/00/00/16/16/index.html
Harnad, S. (2001)
No Easy Way Out. The Sciences 41(2) 36-42.
http://cogprints.soton.ac.uk/documents/disk0/00/00/16/24/index.html
Harnad, S. (2002) Turing Indistinguishability and the Blind
Watchmaker. In: J. Fetzer (ed.) Evolving Consciousness Amsterdam: John Benjamins. Pp 3-18
http://cogprints.soton.ac.uk/documents/disk0/00/00/16/15/index.html
Premack,
D. & Woodruff, G. (1978). Does the chimpanzee have a theory of
mind? Behavioral & Brain Sciences, 4, 515-526.
Shear, J. (Ed.) (1997)
Explaining consciousness : the "hard problem." Cambridge, Mass. :
MIT Press, c1997.
Turing,
A. M. (1950) Computing Machinery and Intelligence. Mind 49:433-460.
http://cogprints.soton.ac.uk/documents/disk0/00/00/04/99/index.html
Whiten,
A. (Ed.) (1991). Natural theories of mind: Evolution, development,
and simulation of everyday
mindreading . Oxford: Blackwell.