Re: Images Vs. Symbols

From: HARNAD Stevan (
Date: Tue Mar 12 1996 - 21:59:58 GMT

> From: "Bollons Nicholas" <>
> Date: Mon, 11 Mar 1996 16:47:49 GMT
> Some form of hypothetical propositional 'language' seems to be a lot
> more plausible explanation than any 'on off' binary system.

A bit of a misunderstanding here: A binary code (for example, morse
code) IS propositional. The issue is about analog images (copies) vs.
arbitrary symbolic codes (computations), not binary vs nonbinary codes.

A binary code can express anything any other arbitrary code can express.
Arbitrary means it doesn't matter what the SHAPE of the symbols is; all
that matters is the rules for manipulating them. Remember the formula
for getting the roots of quadratic equations (X = -b =/ SQRT(b**2-4ac/2a):
It doesn't matter what symbols you use for "X" or "b" or "+": in fact,
it could all be written in binary code, as just 0's and 1's, though it
would be more complicated.

All these differences are just differences in the NOTATIONAL system, the
arbitrary symbols you choose to use. It's exactly the same as with
language: In English we call apples "apples," but we could have called
them "pommes," as in French, or we could have called them "X" or even
"011011000," or whatever it would be in morse code. The shape of the
symbols is arbitrary and does not matter: the symbols neither resemble
nor are causally connected with what they represent (stand for, refer

With analog images, on the other hand, the shape of the image DOES
matter. A picture of an apple resembles an apple; it does not resemble
"apple" or "01101100001."

> There is no way the divergence and mass of sensory input could be
> explained in any 2 dimensional method.

Dimensions don't have anything to do with the binary/nonbinary
distinction. Binary code (0,1) is not 2-dimensional; it just has 2

On the other hand, it is very possible that 2-dimensional"shadows" of
apples on our retinas, and further analog copies of that 2-dimensional
shadow higher up in or brains (the so-called "retinotopic" or analog
copies of the retina that I will talk a little about on Wednesday)
COULD encode all of our diverse sensory input.

A binary code would be symbolic, an analog copy would not be. If your
retina were divided into an on/off grid, the way the screen of a
computer is, then every "shadow" on it could be coded as a string
of 0's and 1's: 0's for every part that is dark, 1 for every part that
is light. Imagine a + shape projected onto a 5x5 grid. There are 1's
in the cells of the grid where the + casts a shadow, 0's everywhere


The rows of this grid could then be made into one very long line.


This line could be used as a binary code for reconstructing the +
on the grid, as long as we (arbitrarily) interpreted the first 0
as representing the upper left corner of the grid, the second 0 as
representing the 2nd..., the last 0 as representing the bottom right

Now look at the original +. Do you notice that right in the middle of
it, there is a 1, with a 1 above, below, left and right of it? Now look
at the long row that represents the grid. What has happened to this
above/below/left/right relationship? It's gone. At least it is no
longer visible (because the long row does not RESEMBLE the +, it merely
REPRESENTS it, as long as we use the arbitrary code, according to
which, say, the sixth binary symbol represents the beginning of the
second row of the grid. (Exercise: Which binary symbol in the long row
represents the middlemost 1 of the grid?)

The long row (also called a "vector") represents the "+" shape
symbolically: It does not resemble the +. There are merely rules that
can be applied to the otherwise arbitrary shape and position of the
binary symbols in the vector so that you can INTERPRET it as the +, and
indeed you can even use it to reconstruct the + if you want to.

Now consider what we would have had if the + had merely cast its shadow
on another surface. The other surface would look very much like the
original +. In particular, there would be a middle-most part of the
shadow that also had above, below, left and right of it parts that were
the same shapes as the parts that were above, below, left and right of
the corresponding middlemost part of the original +.

The shadow is an analog copy of the +, and it preserves the +'s spatial
shape. The binary vector does not preserve the shape, though it
preserves all the information ABOUT the shape (so you could reconstruct
the shape from it, and you could do other operations on it from which
you could figure out things about the shape -- for example, whether or
not the +'s shadow touches the bottom right corner of the grid (the part
that is represented by the last binary digit of the vector). Does it?

The fact is that although a symbolic code is arbitrary in its shape and
an analog copy is not, the symbolic code can DO just about anything the
analog copy can do (though not necessarily as simply or as efficiently).
A picture is worth a thousand words, but if you use enough words, you
can describe as much of a picture as you need to. Words can do
everything pictures can; you may just need a lot more of them to do it.

Can pictures do everything that words can do? The answer is no: The fact
that a picture's shape is not arbitrary, the fact that it resembles what
it represents, is both an advantage and a disadvantage: It allows you to
do certain things much more simply than with symbols (for example,
decide whether or not one shape overlaps with another, or matches it
when rotated); but it does not allow you to SAY anything. Symbols
describe; pictures merely depict. Many things cannot be expressed by
pictures. For example, how would you have explained what I've been
trying to explain in this message, by using only pictures, rather than
symbols, as I did? (I only used a picture once: to illustrate the +

The power of both English and of mathematics and other forms of
computation is the power of symbols: strings of objects (e.g., 0's and
1's) whose shapes are arbitrary but can be used to express anything
that can be expressed in words. To express something in words is to
PROPOSE something. A proposition is a statement, whether in English or
in maths ("the cat is on the mat," "1 + 1 = 2"). Propositions are either
true or false, whereas pictures are neither "true" nor "false," they are
merely pictures. They are not saying anything. (Remember what I said
about Magritte's picture of the pipe and the statement about the pipe?
Pictures are not statements.)

> How, for instance would you
> process hearing and sound ? No 00010's could be used to represent the
> individuality of all the pitches and tones used in music.

Yes they could. A picture (or sound) may be worth a thousand symbols,
or even more, but if you use ENOUGH symbols, you can represent the
sound as close closely as you like. (Think of the Mona Lisa -- in just
black and white, to make it simpler -- represented as a very long
vector of 0's and 1's: Not a single detail would be missing; all the
information about the Mona Lisa would be there. In fact, you could use
the code to make a machine draw the Mona Lisa a million times. It's
just that the code itself, the long vector of symbols, from which the
picture could be drawn, would not itself resemble the Mona Lisa; it
would merely represent it, symbolically.)

So it need not be true that the brain's "code" is analog: It could be
symbolic (propositional), as Pylyshyn suggests. The retina, after all,
is really just a huge grid, and the rods and cones in it could be like
binary 0's and 1's. (They're not, as it turns out, but not because
symbols couldn't have done the job; in computer screens' bit-maps they
DO do the job.)

> A form of 'logical language' (Kosslyn's pseudo English) would work in
> well with the use of A.I in research, as both human and computer
> would seem to use a language for coding input and performing output
> with the language as the medium for both.

Not sure what you mean here. The fact is that Artificial Intelligence
(AI) uses mostly just symbols and rules. And using symbols and rules, it
can do many (perhaps all) the things people can do. This power of
symbolic computation is part of what made people like Pylyshyn and
Fodor conclude that computation is what the brain does too -- that
symbols are the "language of thought." Kosslyn (and Anderson, cited in
Kosslyn's chapter) replied that although symbols COULD do it all, there
seems to be evidence that in people's brains there is analog processing
going on too, and that this is not surprising, because in certain cases
analog processing would be more efficient than symbols and computation.
Shepard's mental rotation (also described in the Kosslyn chapter) is a good

> We all know that a computer does not have a Homunculus (for if it
> did it would also probably have a mind)

It's supremely unlikely that a computer has a mind; so a homunculus is
not a problem for computational explanations of the mind. But does it
have to be a problem for imagistic (analog) explanations? The answer is
that analog processing is also possible without the need of a homunculus
to "see" the picture. A machine could do internal rotation and matching
of analog "shadows" without any more need for a homunculus to "look at"
the internal images than there is a need for a homunculus to understand
the internal symbols in a computer.

> But it does have something
> very similar to one - A Central Processing Unit (C.P.U). This in
> practise processes all the internal information of output and input
> from a variety of different areas.

The computer's CPU is not like a homunculus; it's a purely mechanical
device that is designed to switch circuits, i.e., manipulate 0's and 1's
on the basis of their shapes, much as when you mindlessly calculate the
roots of a quadratic equation. All of that can be done mechanically. No
need for a homunculus at all.

> Could some processing unit exist
> in the brain other than the homunculus ? Processing data in a
> logical and computational way ? Brain Imaging (using P.E.T) has
> identified certain areas that are active during certain functioning
> (retinotopical mapping). This makes the brain a multidimensional organ
> in which different areas perform different functions just like in a
> computer. Could these areas then portray to a C.P.U which processes
> information or data. A logical Homunculus ? Is there any data to
> indicate the physical existence of such a thing ?

You've confused a few things here: Computation is the manipulation
of symbols with arbitrary shapes, on the basis of rules. Retinotopic
maps in the brain are analog projections of the retina, so their shape
is NOT arbitrary. Internal rotation of these analog shapes would not be
symbol manipulation, it would be analog processing.

Dimensions have nothing directly to do with the symbol/image
distinction. The brain does, however, represent 3-dimensional space,
and might do it in an analog or a symbolic way, or both. It also
represents many other dimensions of sensory variation (sounds, smells,
etc.), and, again, could do it either way. If we are to explain HOW our
brain does all those things, and use that to explain our minds,
however, then we need to find an explanation that does NOT require a
homunculus in there, looking at the images or understanding the
symbols, otherwise our work starts all over again, as we try to explain
how IT's mind works...

This archive was generated by hypermail 2b30 : Tue Feb 13 2001 - 16:23:39 GMT