Re: Kosslyn: Mental Imagery

From: Harnad, Stevan (
Date: Sun Jan 14 1996 - 19:58:54 GMT

Wendy has added some points to the summary that Denise and Richard did
not take up:

> From: "Smith, Wendy" <>
> Date: Mon, 27 Nov 1995 09:17:12 GMT
> There are two main issues to the imagery debate:
> 1. What is an image? Is it purely propositional, or partially pictorial?
> 2. Who "looks" at the image, if there is one?
> Kosslyn cites several pieces of evidence suggesting there may be a
> depictive (pictorial) element.
> 1. Maintaining a visual image impairs a contrasting visual
> perception, but facilitates an identical visual perception.
> 2. Imagery can induce the McCullough effect, although only weakly
> (afterimages of the complementary colour)
> 3. Imagery can induce perceptual illusions.
> 4. Times to process evaluations of imaged and perceived patterns
> are similar; scan times of images also correlate with distances.
> 5. Brain scans suggest that the same areas are used for both
> imaging and visual perception.
> 6. Patients with unilateral visual neglect often had a
> corresponding lack of half a mental image.
> 7. Other perceptual deficits have been mirrored in imagery (eg
> someone who can't recognise faces also can't recognise
> faces).
> (N.B. problems with introspective evidence; experimenter expectancy
> - although Kosslyn did try and control for this; using brain damaged
> individuals as evidence for normal functioning; and relating the
> evidence from brain scans to the actual processes in the brain; also
> much of the evidence can support either a pictorial or purely
> propositional explanation).

You're right about the problems with introspective evidence. But the
converging evidence, from many sources, is rather persuasive. Again, it
would be best if the conscious aspect were minimised or relied upon
less, and the burden put on the nature of the performance and the
processing, to show that some of it looks like perception, and that
analog processing might make more sense than symbolic for such tasks.

> However, this evidence led Kosslyn to ask whether visual imagery had
> common mechanisms with visual perception? Kosslyn himself believes
> that not only are images produced by the same processes that are
> responsible for visual perception, but that effective visual
> processing is dependent on the imaging process.
> Visual perception: the high level visual system (ie at the level of
> object recognition) has seven subsystems. These are used in both
> perception and imagery.
> Visual buffer (occipital lobe): input to the rest of the system
> Attention window: selects activity from the visual buffer for
> further processing.
> Ventral system (inferior temporal lobes): receives information from
> the attentional window and encodes shape, colour and texture.
> Dorsal system (posterior parietal lobes): receives information from
> the attention window and encodes spatial properties.
> Associative memory (?posterior, superior temporal lobes): receives
> information from the ventral and dorsal systems. If a single
> representation is not produced, a search is started for the best
> match.
> Information look-up system (dorsolateral prefrontal cortex): based
> on activation from the associative memory, a distinctive
> characteristic of the object is chosen.
> Attention shifting subsystem (frontal, parietal, sub-cortical areas):
> directs the attention window to this characteristic. The ventral
> system is then activated to encode the characteristic. The cycle
> repeats until the object is recognised. This latter activation of
> the ventral system (and activation of the dorsal system) can also
> activate a stored representation, and a mental image results. The
> image can then go throught he same cycles.

Here the evidence for the similarity between the perceptual and imaging
system is suggestive, and should remind you of the Jeannerod (1994)
article about motor imagery and its similarity to motor action and

> Evidence that these systems are involved in imagery:
> Visual buffer: properties affect both perception and imagery in
> similar ways, eg imaging an object to the point of "overflow" - the
> larger the object the greater the distance at overflow.
> Attention window: the smaller the area attended, the easier to
> detect probe dots, in both perceived and imaged patterns.
> Attention shifting subsystem: does not appear involved in imagery ??
> a way of distinguishing image from reality.

There may be a problem distinguishing hallucination and reality, but
images are by definition not so veridical as to be confusable with

> (N.B. back to the problems of introspective evidence etc; also
> control of the image - don't we have a certain amount of control over
> when our images overflow, or whether we can detect dots, which may
> affect the results?)

Yes, and that too distinguishes images from hallucinations. You're
right, though, that this control could make some of the timing effects a
self-fulfilling prophecy. The more the inferences about internal analogs
are based on performance and performance modeling, rather than testimony
about experience, the more compelling they are.

> So, does imagery ride on the back of perception?

It may involve some of the same mechanisms, as in the motor case.
Unlike the motor case, though, sensory perception is passive: Motor
images, like motor acts, are something you might DO. Sensory images are
not things you "do," so "doing" them as mental images is less
straightforward -- but not excluded. But again, inferences from speeded,
involuntary effects are more compelling than the cases where the whole
thing could just be something you do or tell yourself because of a
narrative or scenario about perception and imagery...

> Objects can be recognised from many orientations, and imagery may be
> involved in this.
> When an object presents in a novel way, information from the input
> can be matched to the stored representation, and the object can be
> recognised. But, if there isn't enough information to get a proper
> match, then an image of the next best match is generated for
> comparison to the perceptual image. The generated image isn't so
> much a template, as to provide the missing information. Furthermore,
> as soon as a stored representation is activated (ie as the best match
> inhibits the others) it begins to send an image for use as feedback
> within this system.

I would only believe this if a highly effective and upwardly scaling
model recommended it as the better way to get things done; I would
certainly never accept it from introspective testimony about "how" one
was doing it, or even from timing data.

> To return to the original questions:
> 1. What is an image? Is it purely propositional, or partially pictorial?
> 2. Who "looks" at the image, if there is one?
> Kosslyn suggests:
> 1. It has pictorial elements, although it may be processed propositionally.
> 2. It is part of the perceptual process, and the same mechanisms
> involved in perception "look at" (process) the image.

The real question is: What can a mechanism like this actually do? Are
there things it would do better than a symbol manipulating mechanism?
Would it scale up better? these questions remain to be answered. These
timing data just scratch the surface.



ABSTRACT:This target article concerns how motor actions are neurally represented and coded. Action planning and motor preparation can be studied using motor imagery. A close functional equivalence between motor imagery and motor preparation is suggested by the positive effects of imagining movements on motor learning, the similarity between the neural structures involved, and the similar physiological correlates observed in both imagining and preparing. The content of motor representations can be inferred from motor images at a macroscopic level: from global aspects of the action (the duration and amount of effort involved) and from the motor rules and constraints which predict the spatial path and kinematics of movements. A microscopic neural account of the represenation of object-oriented action is described. Object attributes are processed in different neural pathways depending on the kind of task the subject is performing. During object-oriented action, a pragmatic representation is activated in which object affordances are transformed into specific motor schemata independently of other tasks such as object recognition. Animal as well as clinical data implicate posterior parietal and premotor cortical areas in schema instantiation. A mechanism is proposed that is able to encode the desired goal of the action and is applicable to different levels of representational organization.

This archive was generated by hypermail 2b30 : Tue Feb 13 2001 - 16:23:57 GMT