Figure 1.
As cognitive science, including especially cognitive neuroscience, closes in on the first realistic models of the human mind, philosophical puzzles and problems that have been conveniently postponed or ignored for generations are beginning to haunt the efforts of the scientists, confounding their vision and leading them down hopeless paths of theory. I will illustrate this claim with a brief look at several temporal phenomena which appear anomalous only because of a cognitive illusion: an illusion about the point of view of the observerix. Since there is no point in the brain where "it all comes together," several compelling oversimplifications of traditional theorizing must be abandoned.
Wherever there is a conscious mind, there is a point of view.Endnote 1 This is one of the most fundamental ideas we have about minds--or about consciousness. A conscious mind is an observer, who takes in a limited subset of all the information there is: an observer takes in the information that is available at a particular (roughly) continuous sequence of times and places in the universe. For most practical purposes, we can consider the point of view of a particular conscious subject to be just that: a point moving through space-time. We explain the startling time gap between the sound and sight of the distant fireworks by noting the different transmission speeds of sound and light. They arrive at the observer (at that point) at different times, even though they left the source at the same time.
What happens, though, when we close in on the observer, and try to locate the observer's point of view more precisely, as a point within the individual? The simple assumptions that work so well on larger scales begin to break down. There is no single point in the brain where all information funnels in, and this fact has some far from obvious--indeed quite counterintuitive--consequences.
Descartes, one of the first to think seriously about the what must happen once we look closely inside the body of the observer, elaborated an idea that is so superficially natural and appealing that it has permeated our thinking about consciousness ever since. Descartes decided that the brain did have a center: the pineal gland, which served as the gateway to the conscious mind (see figure 1).
The pineal gland is the only organ in the brain that is in the midline, rather than paired, with left and right versions. Smaller than a pea, it sits in splendid isolation on its stalk, attached to the rest of the nervous system just about in the middle of the back of the brain. Since its function was quite inscrutable (and it still is), Descartes proposed a role for it: in order for a person to be conscious of something, traffic from the senses had to arrive at this station, where it thereupon caused a special--indeed magical--transaction to occur between the person's material brain and immaterial mind. Descartes' vision of the pineal's role as the turnstile of consciousness (we might call it the Cartesian bottleneck) is hopelessly wrong. But while materialism of one sort or another is now the received opinion among scientists and philosophers alike, with a few eminent exceptions, even the most sophisticated materialists today often forget that once Descartes' ghostly res cogitans is discarded, there is no longer a role for a centralized gateway, or indeed for any functional center to the brain. The pineal gland is not only not the FAX machine to the Soul; it is also not the Oval Office of the brain, and neither are any of the other portions of the brain. The brain is Headquarters, the place where the ultimate observer is, but there is no reason to believe that the brain itself has any deeper headquarters, any inner sanctum arrival at which is the necessary or sufficient condition for conscious experience. In short, there is no observer inside the brain.
Light travels much faster than sound, as the fireworks example reminds us, but we now know that it takes longer for the brain to process visual stimuli than to process auditory stimuli. As Ernst Pöppel (1985, 1988) has pointed out, thanks to these counterbalancing differences, the "horizon of simultaneity" is about 10 meters: light and sound that leave the same point about 10 meters from the observer's sense organs produce neural responses that are "centrally available" at the same time. Can we make this figure more precise? There is a problem. The problem is not just measuring the distances from the external event to the sense organs, or the transmission speeds in the various media, or allowing for individual differences. The more fundamental problem is deciding what to count as the "finish line" in the brain.
Pöppel obtained his result by comparing behavioral measures: mean reaction times (button-pushing) to auditory and visual stimuli. The difference ranges between 30 and 40 msec, the time it takes sound to travel approximately 10 meters (the time it takes light to travel 10 meters is insignificantly different from zero). Pöppel used a peripheral finish line--external behavior--but our natural intuition is that the experience of the light and sound happens between the time the vibrations hit our sense organs and the time we manage to push the button signalling that experience. And it happens somewhere centrally, somewhere in the brain on the excited paths between the sense organ and the finger. It seems that if we could say exactly where, we could say exactly when the experience happened. And vice versa: if we could say exactly when it happened, we could say where in the brain conscious experience was located.
Let's call the idea of such a centered locus in the brain Cartesian materialism, since it's the view you arrive at when you discard Descartes' dualism but fail to discard the imagery of a central (but material) Theater where "it all comes together". Perhaps no one explicitly endorses Cartesian materialism, and no doubt most theorists would insist that they have explicitly rejected such an obviously bad idea. But the persuasive imagery of the Cartesian Theater, in its materialistic form, keeps coming back to haunt them.
This picture of how conscious experience must sit in the brain is a natural extrapolation of the familiar and undeniable fact that for macroscopic time intervals, we can indeed order events into the categories "not yet observed" and "already observed" by locating the observer and plotting the motions of the vehicles of information relative to that point. But when we try to extend this method to explain phenomena involving very short time intervals, we encounter a logical difficulty: If the "point" of view of the observer is spread over a rather large volume in the observer's brain, the observer's own subjective sense of sequence and simultaneity must be determined by something other than "order of arrival" since order of arrival is incompletely defined until we specify the relevant destination. If A beats B to one finish line but B beats A to another, which result fixes subjective sequence in consciousness? (cf. Minsky, 1986, p.61) Which point or points of "central availability" would "count" as a determiner of experienced order, and why?
Doesn't it follow as a matter of geometric necessity that our conscious minds are located at the termination of all the inbound processes, just "before" the initiation of all the outbound processes that implement our actions? Advancing from one periphery along the input channels, we ascend though the optic nerve, and up through various areas of the visual cortex, and then . . . ? Advancing from the other periphery by swimming upstream from the muscles and the motor neurons that control them, and then up into the supplementary motor area in the cortex and then . . . ? These two journeys advance towards each other up two slopes, the afferent and the efferent. However difficult it might be to determine in practice the precise location of the Continental Divide in the brain, must there not be, by sheer geometric extrapolation, a highest point, a turning point, a point such that all tamperings on one side of it are pre-experiential, and all tamperings on the other are post-experiential?
In Descartes's picture, this is obvious to visual inspection, since everything funnels to and from the pineal station. It might seem, then, that if we were to take a more current model of the brain, we should be able to color-code our explorations, using, say, red for afferent or inbound and green for efferent or outbound; wherever our colors suddenly changed would be a functional midpoint on the great Mental Divide.
This curiously compelling argument may well ring a bell. It is the twin of an equally spurious argument that has recently been all too influential: Arthur Laffer's notorious Curve, the intellectual foundation (if I may speak loosely) of Reaganomics.
If the government taxes at 0% it gets no revenue, and if it taxes at 100% no one will work for wages, so it gets no revenue; at 2% it will get roughly twice the revenue as at 1%, and so forth, but as the rate rises, diminishing returns will set in; the taxes will become onerous. Looking at the other end of the scale, 99% taxation is scarcely less confiscatory than 100%, so scarcely any revenue will accrue; at 90% the government will do better, and better still at the more inviting rate of 80%. The particular slopes of the curve as shown may be off, but mustn't there be, as a matter of geometric necessity, a place where the curve turns, a rate of taxation that maximizes revenue? Laffer's idea was that since the current tax rate was on the upper slope, lowering taxes would actually increase revenues. It was a tempting idea; it seemed that it just had to be right. But as Martin Gardner (1981) has pointed out, just because the extreme ends of the curve are clear, there is no reason why the unknown part of the curve in the middle regions has to take a smooth course.
In a satiric mood, he proposes the alternative "neo-Laffer Curve," which has more than one "maximum," and the accessibility of any one of them depends on complexities of history and circumstance that no change of a single variable can possibly determine. We should draw the same moral about what lies in the fog inboard of the afferent and efferent peripheries: The clarity of the peripheries gives us no guarantee that the same distinctions will continue to apply all the way in.
The "technosnarl" Gardner envisages for the economy is simplicity itself, in fact, compared to the jumble of activities occurring in the more central regions of the brain. So we must stop thinking of the brain as if it had such a central point. This is not just an innocuous shortcut; it is a bad habit. In order to break this bad habit of thought, we need to explore some instances of the bad habit in action, but we also need a good image with which to replace it.
Here is a first version of the good replacement, the Multiple Drafts model of consciousness--but I expect it will seem quite alien and unlikely at first. That's how entrenched the Cartesian Theater idea is. According to the Multiple Drafts model, all varieties of perception, and indeed all varieties of thought and action, are accomplished in the brain by multi-track processes of interpretation and elaboration that occur over large fractions of a second, during which time various additions, incorporations, emendations, and overwritings of content can occur, in various orders. Feature-detections or discriminations only have to be made once. That is, once a particular "observation" of some feature has been made, by a specialized, localized portion of the brain, the information content thus fixed does not have to be sent somewhere else to be rediscriminated by some "master" discriminator. In other words, it does not lead to a re-presentation of the already discriminated feature for the benefit of the audience in the Cartesian Theater.
These spatially and temporally distributed content-fixations are themselves precisely locatable in both space and time, but their onsets do not mark the onset of consciousness of their content. It is always an open question whether any particular content thus discriminated will eventually appear as an element in conscious experience. These distributed content-discriminations yield, over the course of time, something rather like a narrative stream or sequence, which can be thought of as subject to continual editing by many processes distributed around in the brain, and continuing indefinitely into the future. This stream of contents is only rather like a narrative because of its multiplicity; at any point in time there are multiple "drafts" of narrative fragments at various stages of "editing" in various places in the brain. Probing this stream at different intervals produces different effects, precipitates different narratives from the subject. If one delays the probe too long (overnight, say) the result is apt to be no narrative left at all--or else a narrative that has been digested or "rationally reconstructed" until it has no integrity. If one probes "too early", one may gather data on how early a particular discrimination is achieved in the stream, but at the cost of disrupting the normal progression of the stream. Most importantly, the Multiple Drafts model avoids the tempting mistake of supposing that there must be a single narrative (the "final" or "published" draft) that is canonical--that is the actual stream of consciousness of the subject, whether or not the experimenter (or even the subject) can gain access to it.(For a similar model, see William Calvin's (1990) model of consciousness as "scenario-spinning".)
Suppose I tamper with your brain, inserting in your memory a bogus lady in a hat where none was (e.g., at the party on Sunday). If on Monday, when you recall the party, you remember her, and can find no internal resources for so much as doubting the veracity of your memory, we would say that you never did experience her; that is, not at the party on Sunday.
Of course your subsequent experience of (bogus) recollection can be as vivid as may be, and on Tuesday we can certainly agree that you have had vivid conscious experiences of there being a lady in a hat at the party, but the first such experience, we would insist, was on Monday, not Sunday (though it doesn't seem this way to you).
We lack the power to insert bogus memories by neurosurgery, but sometimes our memories play tricks on us, so what we cannot yet achieve surgically happens in the brain on its own. Sometimes we seem to remember, even vividly, experiences that never occurred. Let's call such post-experiential contaminations or revisions of memory Orwellian, after George Orwell's chilling vision in the novel 1984 of the Ministry of Truth, which busily rewrote history and thus denied access to the (real) past to all who followed.
Orwellian revision is one way to fool posterity. Another is to stage show trials, carefully scripted presentations of false testimony and bogus confessions, complete with simulated evidence. Let's call this ploy Stalinesque. Notice that if we are usually sure which mode of falsification has been attempted on us, the Orwellian or the Stalinesque, this is just a happy accident. In any successful disinformation campaign, were we to wonder whether the accounts in the newspapers were Orwellian accounts of trials that never happened at all, or true accounts of phony show trials that actually did happen, we might be unable to tell the difference. If all the traces--newspapers, videotapes, personal memoirs, inscriptions on gravestones, living witnesses, etc.--have been either obliterated or revised, we will have no way of knowing whether a fabrication happened first, culminating in a staged trial whose accurate history we have before us, or rather, after a summary execution, history-fabrication covered up the deed: no trial of any sort actually took place.
The distinction between reality and (subsequent) appearance, and the distinction between Orwellian and Stalinesque methods of producing misleading archives, work unproblematically in the everyday world, at macroscopic times scales. One might well think they apply unproblematically all the way in, but this is the illusion, and we can catch it in the act in a thought experiment that differs from the first one in nothing but time scale.
Suppose you are standing on the corner and a long-haired lady dashes by. About one second after this, a subterranean memory of some earlier long-haired lady--wearing eyeglasses--contaminates the memory of what you have just seen: when asked a minute later for details of the lady you just saw, you report, sincerely but erroneously, her eyeglasses. Just as in the case of the lady in the hat, we are inclined to say that your original visual experience, as opposed to the memory of it seconds later, was not of a woman with glasses. But due to the subsequent memory-contaminations, it seems to you exactly as if at the first moment you saw her, she surprised you with her eyeglasses. An Orwellian revision has happened: there was a fleeting instant, before the memory contamination took place, when it didn't seem to you she had glasses on. For that brief moment, the reality of your conscious experience was a long-hairedlady without eyeglasses, but this historical fact has become inert; it has left no trace, thanks to the contamination of memory that came one second after you glimpsed her.
This understanding of what happened is jeopardized, however, by an alternative account. Your subterranean earlier memories of that long-haired lady with the eyeglasses could just as easily have contaminated your experience on the upward path, in the processing of information that occurs "prior to consciousness" so that you actually hallucinated the eyeglasses from the very beginning of your experience.
In that case, your obsessive memory of the lady with the glasses would be playing a Stalinesque trick on you, creating a show trial in experience, which you then accurately recall at later times, thanks to the record in your memory. To naive intuition these two cases are as different as can be: told the first way (figure 9) you suffer no hallucination at the time the lady dashes by, but suffer subsequent memory-hallucinations: you have false memories of your actual ("real") experience. Told the second way (figure 10) you hallucinate when she runs by, and then accurately remember that hallucination (which "really did happen in consciousness") thereafter. Surely these are distinct possibilities no matter how finely we divide up time?
No. Here the distinction between perceptual revisions and memory revisions that works so crisply at other scales is not guaranteed application. We have moved into the foggy area (the control window) in which the subject's point of view is spatially and temporally smeared, and the question Orwellian or Stalinesque? need have no answer.
There is a temporal window that began when the long-haired lady dashed by, exciting your retinas, and ended when you expressed--to yourself or someone else--your eventual conviction that she had on eyeglasses. At some time during this interval, the content eyeglasses was spuriously added to the content you expressed. We may assume (and might eventually confirm in detail) that there was a brief time when the content long-haired lady had already been discriminated in the brain but before the eyeglasses content had been erroneously "bound" to it. Indeed, it would be plausible to suppose that this discrimination of a long-haired lady was what triggered the memory of the earlier lady with the eyeglasses. What we would not know, however, is whether this spurious binding was before or after the fact--the presumed fact of "actual conscious experience". Were you first conscious of a lady without eyeglasses and then conscious of a lady with eyeglasses, a subsequent consciousness which wiped out the memory of the earlier experience, or did the very first instant of conscious experience already spuriously contain the glasses? If Cartesian materialism were true, this question would have an answer, even if we--and you--could not determine it retrospectively by any test. But when we abandon Cartesian materialism (as I think we should--and most theorists would agree) the distinction between pre-experiential and post-experiential content revisions cannot always be maintained. I will illustrate this with two simple cases.
Color phi. If two or more small spots separated by as much as 4 degrees of visual angle are briefly lit in rapid succession, a single spot will seem to move. The philosopher Nelson Goodman asked Paul Kolers whether the phi phenomenon persisted if the two illuminated spots were different in color, and if so, what happened to the color of "the" spot as "it" moved? The answer, when Kolers and von Grünau performed the experiments, was striking: the spot seems to begin moving and then change color abruptly in the middle of its illusory passage toward the second location. Goodman wonders: "how are we able . . .to fill in the spot at the intervening place-times along a path running from the first to the second flash before that second flash occurs? "(1978, p.73) (The same question can of course be raised about any phi, but the color-switch in mid-passage vividly brings out the problem.) Unless there is precognition in the brain, the illusory content cannot be created until after some identification of the second spot occurs in the brain. But if this identification of the second spot is already "in conscious experience" would it not be too late to interpose the illusory color-switching-while-moving scene between the conscious experience of spot 1 and the conscious experience of spot 2? How does the brain accomplish this sleight-of-hand?
Consider, first, a Stalinesque mechanism: in the brain's editing room, located before consciousness, there is a delay, a loop of slack, as it were, like the "tape delay" used in broadcasts of "live" programs which gives the censors in the control room a few seconds to bleep out obscenities before broadcasting the signal. In the editing room, first frame A, of the red spot, arrives, and then, when frame B, of the green spot, arrives, some interstitial frames (C and D) can be created and then spliced into the film (in the order A,C,D,B) on its way to projection in the theater of consciousness. By the time the "finished product" arrives at consciousness, it already has its illusory insertion.
Alternatively, there is an Orwellian mechanism: shortly after the consciousness of the first spot and the second spot (with no illusion of apparent motion at all), a revisionist historian of sorts, in the brain's memory-library receiving station, notices that the unvarnished history of this incident doesn't make enough sense, so he "interprets" the brute events, red-followed-by-green, by making up a narrative about the intervening passage, complete with midcourse color change, and installs this history, incorporating his glosses, frames C and D (in figure 7), in the memory library for all future reference. Since he works fast, within a fraction of a second--the amount of time it takes to frame (but not utter) a verbal report of what you have experienced--the record you rely on, stored in the library of memory, is already contaminated. You say and believe that you saw the illusory motion and color change, but that is really a memory hallucination, not an accurate recollection of your original consciousness.
Compare the question here with the question we asked about the lady and the eyeglasses: This time we ask: was the subject conscious of the midcourse color-switch before being conscious of the green spot, or did the color-switch get inserted retrospectively, after consciousness of the green spot had already occurred?
How could we see which of these hypotheses is correct? It might seem that we could rule out the Stalinesque hypothesis quite simply, because of the delay in consciousness it postulates. In Kolers and von Grünau's experiment (1976), there was a 200msec difference in onset between the red and green spot, and since, ex hypothesi, the whole experience cannot be composed by the editing room until after the content green spot has reached the editing room, consciousness of the initial red spot will have to be delayed by at least that much. (If the editing room sent the content red spot up to the theater of consciousness immediately, before receiving frame B and then fabricating frames C and D, the subject would presumably experience a gap in the film, a noticeable delay of around 200msec between A and C).
Suppose we ask subjects to press a button "as soon as you experience a red spot," and we find little or no difference in response time to a red spot alone versus a red spot followed 200msec later by a green spot (in which case the subjects report color-switching apparent motion). This could be because there is always a delay of at least 200msec in consciousness, but aside from the biological implausibility of such a squandering of time, there is the evidence from many quarters that responses under conscious control, while slower than such responses as reflex blinks, occur with close to the minimum latencies that are physically possible; after subtracting the demonstrable travel times for incoming and outgoing pulse trains, and the response preparation time, there is little time left over in "central processing" in which to hide a 200msec delay. So the responses had to have been initiated before the discrimination of the second stimulus, the green spot. This would seem overwhelmingly to favor the Orwellian, post-experiential mechanism: as soon as the subject becomes conscious of the red spot, he initiates a button-press. While that button press is forming, he becomes conscious of the green spot. Then both these experiences are wiped from memory, replaced in memory by the revisionist record of the red spot moving over and then turning green halfway across. He readily and sincerely (but falsely) reports having seen the red spot moving towards the green spot before changing color.
If the subject insists that he really was conscious from the very beginning of the red spot moving and changing color, the Orwellian theorist will firmly explain to him that he is wrong; his memory is playing tricks on him; the fact that he pressed the button when he did is conclusive evidence that he was conscious of the (stationary) red spot before the green spot had even occurred. After all, his instructions were to press the button when he was conscious of a red spot. He must have been conscious of the red spot about 200msec before he could have been conscious of it moving and turning green. If that is not how it seems to him, he is simply mistaken.
The defender of the Stalinesque (pre-experiential) alternative is not defeated by this, however. Actually, he insists, the subject responded to the red spot before he was conscious of it! The directions to the subject (to respond to a red spot) had somehow trickled down from consciousness into the editing room, which unconsciously initiated the button-push before sending the edited version (frames ACDB) up to consciousness for "viewing". The subject's memory has played no tricks on him; he is reporting exactly what he was conscious of, except for his insistence that he consciously pushed the button after seeing the red spot; his "premature" button-push was unconsciously (or preconsciously) triggered.
Where the Stalinesque theory postulates a button-pushing reaction to an unconscious detection of a red spot, the Orwellian theory postulates a conscious experience of a red spot that is immediately obliterated from memory by its sequel. So here is the rub: we have two different models of what happens in the phi phenomenon: one posits a Stalinesque "filling in" on the upward, pre-experiential path, and the other posits an Orwellian "memory revision" on the downward, post-experiential path, and both of them are consistent with whatever the subject says or thinks or remembers. Note that the inability to distinguish these two possibilities does not just apply to the outside observers who might be supposed to lack some private data to which the subject had "privileged access". You, as a subject in a phi phenomenon experiment, could not discover anything in the experience from your own first-person perspective that would favor one theory over the other; the experience would "feel the same" on either account.
Today we have grown quite comfortable with the distinction between the spatial location in the brain of the vehicle of experience, and the location "in experiential space" of the item experienced. In short we distinguish representing from represented, vehicle from content. We have grown sophisticated enough to recognize that the products of visual perception are not, literally, pictures in the head even though what they represent is what pictures represent well: the layout in space of various visible properties. What we tend to miss is that we should make the same distinction for time: when in the brain an experience happens must be distinguished from when it seems to happen. Indeed, one way of looking at the message of my talk is as a straightforward extension of the common wisdom about experience of space to experience of time; the representation of space in the brain does not always use space-in-the-brain to represent space, and the representation of time in the brain does not always use time-in-the-brain. In fact, it cannot always use time-in-the-brain because of the lack of a single finish line, or Cartesian theater, where "order of arrival" fixes subjective order in consciousness.
Calvin, W., 1990, The Cerebral Symphony: Seashore Reflections on the Structure of Consciousness, New York: Bantam
Gardner, M., "The Laffer curve and other laughs in current economics," Scientific American, December, 1981, vol.245, pp. 18-31.
Goodman, N., 1978, Ways of Worldmaking, Hassocks, Sussex: Harvester.
Kolers, P. and von Grünau, M., 1976, "Shape and color in apparent motion," Vision Research, 16, pp.329-35.
Pöppel, E., 1985, Grenzen des Bewusstseins, Stuttgart: Deutsche Verlags-Anstal, translated as Mindworks: Time and Conscious Experience, New York: Harcourt Brace Jovanovich, 1988.
1. Material is this article is drawn from my book, Consciousness Explained, Boston: Little Brown, 1991.