AISB Convention 7-11 April 2003 University of Aberystwyth Wales
Abstract
Language is a skilled activity. In the development and acquisition of the skill, imitation may play different roles. Imitation in language may be related to and throw light on the role and functioning of imitation in other areas including imitation in robotics. What part does imitation play in the child’s acquisition of its mother language? What role did imitation play in the evolutionary origin and diversification of language? How much has imitation to do with the sources of the words we use and the ways those words are put together? These questions can be considered at different levels, the surface forms of language and speech, the underlying systematicies of language and speech, the problem of speech at the articulatory level and beyond or beneath that the problem of the functioning of imitation at the neural level. Imitation of any kind involves a relation between motor and perceptual functioning, between the motor system of the brain and the visual and other sensory systems. Language and speech also require interaction and coordination between motor activity and perceptual activity. The role and functioning of imitation in language and speech are subjects of study in many different disciplines, not only linguistics proper but also child development, neurology, evolutionary theory, social psychology. A central idea in this paper is a new emphasis on the bodily basis of language in relation to imitated speech and gesture, and more specifically on cerebral motor organisation as providing a possible new approach to the symbol-grounding problem.
This paper comes basically from a linguistics perspective, though much expanded beyond current mainline linguistics. Language is what makes us human. It has been the subject of massive discussion and research for hundreds and thousands of years, and now in many disciplines. A first distinction is between language and speech. Speech is concerned with the human mechanisms in exploiting language; language is the whole framework, individual and social, within which language functions, a distinction perhaps parallel to Chomsky’s competence and performance. Until comparatively recently language has been a matter for linguists, with all the multitudinous complexities of the subject: historical development, phonology, semantics, syntax, manifested in a plethora of grammars, syntactic theories, semantic theories, together with the over-riding complexity resulting from the vast number of different actual languages, different lexicons, different grammatical systems. Now there is concern with language in many other disciplines; in philosophy there has been what Rorty has termed “the linguistic turn”, linguistic philosophy and philosophy of language. In linguistics proper there has been the massive development of syntactic theory associated with Chomsky but also involving many other linguists, in the process making linguistics an impenetrable jungle for non-linguists. In psychology speech and language have generated a large experimental literature, not surprisingly given the importance of language in the functioning of the human mind and in society. In evolutionary biology there has been a widening study of language, starting with the seminal 1975 New York Academy of Sciences Conference on language origin. The development of new technology (PET, fMRI, TMS, MEG) allowing direct inspection of brain functioning has meant that neurology has been able to give a much more concrete content to its concern with language, beyond traditional research on the speech pathologies. It is not necessary to emphasise AI’s focus, from the very beginning, on natural language – where less progress has been made than was once hoped. As regards imitation, this has been until recently a neglected and troublesome topic; it is now becoming a major field in its own right; as for language, research now is cutting across many different disciplines. Perhaps the 2002 Royaumont Conference will be seen as imitation’s seminal event parallel to the 1975 language conference. The well-publicised discovery of mirror neurons by Rizzolatti, Gallese and colleagues, together with other rapidly advancing neuroscience theory and research into perception and motor control, now opens up the possibility for joint examination of these two major fields of human functioning, language and imitation. The paper first brings together what has been said about the relation between imitation and language by practitioners in linguistics, psychology, neurology and robotics. It then considers the relevance of advances in neuroscience bearing on imitation and language and how they may throw light on the developmental as well as the evolutionary acquisition of language, in terms of the integral functioning of language, perception and action. This leads on to an account of motor theory of language origin and function (extending beyond the well-known motor theory of speech perception) and the ways in which this permits practical demonstration of the non-arbitrary, non-conventional foundations and operations of language.
The issue is the role that imitation plays in the origin of language, in the acquisition of language by children, in the historical development of languages, and in the social uses of language. There are separate sub-questions about imitation in relation to phonology, lexicon and syntax. Not surprisingly views differ sharply, not only between the different disciplines but also within disciplines, notably within linguistics, and in the evolutionary account of language. Much turns on whether the origin, source, acquisition and functioning of language are seen as innately based or as the product of general human cognitive capacities.
Most linguists do not believe that learning a first language is anything like the kind of controlled and directed learning that is involved in acquiring skills or a second language in an adult. All agree in judging the acquisition of a child’s first language to be one of the mysteries of human life, effortless in virtually any environment(Lieberman 1991). Bloomfield(1933) said the acquisition by every child of the group’s habits of speech in the first years of life is no doubt the greatest intellectual feat but exactly how children learn to speak is not known. For Lenneberg(1967) the construction of proper sentences is not facilitated by telling a child how to do it; no grammar, new or old, could help an essentially language-deficient person to put words together to form good sentences. For Jean Aitchison (1989), forcing children to imitate is a dismal failure; children cannot be trained like parrots. Chomsky (1988) says that language acquisition commonly proceeds on course even without any concern on the part of the human models; the precision of phonetic detail a child acquires cannot possibly be the result of training; the speed and precision of vocabulary growth has to be explained by a biological endowment for language; the child somehow has the concepts available before experience with language and basically learns labels for already existing concepts [not very plausible]. Pinker (1995) asks ‘what in the world is a word?’, and judges it to be a pure symbol’ [a verbal, not a substantive answer]. Deacon, in his masterly book The Symbolic Species (1997), contradicts Chomsky; there are no innate symbolic categories; the primary mystery is the origin of symbolic abilities; what has kept other species from evolving languages is just the simple problem of figuring out how combinations of words refer to things, the curious difficulty of symbolic reference. For Jackendoff, in his recent (2002) book on the foundations of language, the effortlessness of vocabulary acquisition is a human adaptation, the capacity for an open vocabulary being independent of that for grammatical adaptation. So where does this leave the question of the significance of imitation in language acquisition and indeed in language origin? It seems clear that imitation does not operate in any deliberate way through training children to acquire words, grammatical rules and speech sounds, as adults have to do in learning a second language. At best it seems to be a matter of unconscious imitation but there remain many obscurities about how young children extract from the stream of speech sound the ‘right’ words and the ‘right’ grammatical rules. If the capacity for imitation is innate, and the language capacity is also innate, then examining imitation more closely may provide a way forward; there are further questions about the exact nature of the human capacity for imitation and how it developed in human evolutionary history. There are specific questions how imitation can operate in the acquisition of the distinct aspects of language: phonology, lexicon and syntax.
From Trevarthen (1984, 1994) , infants learn by imitation but the structural foundations for imitative movements must be innate; some infants display remarkable precision in imitation from birth but there are large individual differences; around six months an infant can be observed imitating signal gestures and mannerisms which have the characteristics of ‘protosigns’ in proto-conversations, with a shared grammar of action. For Gopnik, Meltzoff and Kuhl (1999) the ability to imitate is actually amazing.; newborns have never seen their own face and in order to imitate must somehow understand the similarity between an internal feeling and the external face they see; babies spontaneously coordinate their own expressions, gestures, and voices with the expressions, gestures and voices of other people; the problem of language is the mysterious gap between the sound waves that actually reach our ears and the sounds and words we create in our minds; we remain faced with the central problem of language, learning what words mean; Augustine’s theory that children learn language by associating a name and a thing turns out to be wrong. Williams, Whiten, Suddendorf and Perrett (2001) say imitation involves converting an action plan from the other’s perspective into one’s own; autistic children may have a specific deficit in motor imitation, which, curiously, may go with echolalia and other repetitive behaviours; autistic children may suffer from a failure or distortion in the development of the neural mirror system. In Meltzoff and Prinz’s (2002) developmental theory of imitation, the observer uses visual perception as the basis for an action plan; execution of the motor output involves vision, cross-modal coordination and motor control; research shows that the speech code, mapping sound to sign , is not exclusively auditory or motor, but multimodal; infants become capable of delayed imitation (execution separated in time from observation), a possible escape route from echolalia and a first step towards symbol development. What seems clear from the above is that the intermodal or amodal relationship of perception and action provides the basis for imitation. This also seems to be the case for the relation between speech production and speech perception, and for the relation between gesture and speech.
Hurley’s useful account(1998) drew attention to the new understanding in neuroscience of the relation between perception and action; recent neurophysiological evidence at both single-cell and cell population levels suggested a shared coding for perception and action (as in the motor theory of speech perception). Decety and his colleagues (2000) summarised usefully the numerous studies in neurophysiology which lead one to postulate a functional equivalence between producing an action, imaging it, verbalising it and observing it, an attractive model offering a parsimonious explanation of the cognitive mechanisms underlying the generation of an intentional action and the recognition of actions performed by others, at the same time compatible with an evolutionary approach to the development of the cognitive capacities; neurophysiological studies with monkeys gave support to the idea of a possible close relation between gesture and language and between the neural structures involved in the production of language and motor control. Research (Iacoboni et al. 1999) on cortical mechanisms of human imitation suggested that language perception could be based on a direct matching between linguistic material and the motor actions responsible; the minimal brain regions for imitation belong to the area of the cerebral cortex known to be important for language; the neural mechanisms implementing imitation might be used also for language; similarities between the structure of actions and the structure of sentences reinforced this notion. The discussion of imitation also has a relation to the well-established approach of ideomotor theories and ideomotor actions (Lotze and William James). Mirror neurons research is advancing rapidly and is of increasing significance for many areas of human functioning.
Attempts to endow robots with imitation capabilities are quite recent and those aimed at studying biological imitation through robotics are few. There are divergent views about the innate or social nature of imitation and its relevance for language acquisition. This roboticist conflation draws on Billard and Dautenhahn 1999, Billard 1999, Breazeal and Scassellati 2000, Weber, Mataric and Jenkins 2000, Mataric 2000. Billard and Mataric 2001, Jenkins and Maratic 2001, Mataric 2002. Social interaction can be a powerful way for transferring important skills, tasks and information to a robot. However, imitation for robots presents its own unique set of research questions, each a complex problem which the robotics community has only begun to address: how does the robot know when to imitate? how does the robot know what to imitate? how does the robot map observed actions into behavioural responses? Knowing what to imitate is fundamentally a problem of determining saliency: which objects become the target of attention. For actions like facial expression, the robot is not capable of observing its own motor behaviours directly; the mapping must either be innate or be learned using an external reinforcement source. Constructing the necessary perceptual, motor, and cognitive skills is extremely difficult as also is the integration of each of these components. Some research aims to facilitate the learning of language using an imitative algorithm, root learning of a small lexicon and elementary syntax, a non-nativist approach. Imitation involves the interaction of perception, memory and motor control; to keep the model behaviourally relevant, it is based and constrained with data from neuroscience, psychophysics and ethology, integrating findings from neuroscience and particularly from research into mirror neurons. Perceptual and motor routines are combined with the function of mirror neurons into primitives; perceptual-motor primitives, extracted manually or automatically from human movement data, are used as a basis set of motion, serving as a vocabulary for classifying and imitating observed movements, studying their role in motor control and their effect on perception and learning.
From the material above (only a small selection from the massive research and other literature) what emerges relevant to the theme of this paper and more specifically for the evolutionary acquisition of language and child acquisition and for the functioning of a complex and expanding community language?
1. much uncertainty and disagreement about the nature of imitation. Imitation as it
functions in language may be more like what some advocates of ‘true’ imitation
dismissively term ‘contagion’. If we could explain why we yawn when we see someone
yawn, this would be an important advance;
2. no consensus about how humans acquired
the capacity to represent things by words whilst other species did not;
3. innate
imitation rather than training or instruction may have the major role in child
acquisition of language, more certainly for phonology than for lexicon or syntax. It
remains unclear how imitation makes possible the linking of word meaning and word
sound or the acquisition of the complexities of the grammatical system;
4. it is not
clear in what sense the functioning of language socially depends in any way on
imitation;
5. a widening recognition that what is important for imitation, language
and speech is an intimate coordination between brain motor function and the sensory
systems, that is, how perception in all its forms is tied to motor action,
dependent on cross-modal, amodal or multimodal operations in the brain;
6. the
potential significance for the relation between imitation and language of the discovery
of mirror neurons and motor primitives (to which should be added research into motor
imagery and motor equivalence)
7. very considerable problems remaining in robotics
but some indications of a move under way from computational to biologically inspired
robotics.
There is a sense that biological linguistics, language research, neuroscience and biologically-based robotics are beginning to converge. We may reasonably look for more answers from continued neuroscience advances. The next section reports some recent research findings possibly relevant for language.
Imitation clearly involves a close and complex relation between the neural basis for movement and the neural basis for perception. The same is true for language which, as manifested in speech, is an intensely motoric system,; the motor control segments of the brain have to organise the very complex processes involved in generating speech sound from the articulatory organs, requiring neuromotor control of the elaborate musculature of the tongue, the lips, the larynx and the respiratory system. Motor control in neuroscience is a central field of research. Though no definitive or complete account as yet exists, an outline of the way in which the brain organises and controls the execution of all types of bodily action, including articulatory action, is gradually becoming apparent.
The discovery of mirror neurons has been well-publicised and is being applied in many research projects. The basic discovery is the existence of neurons in the monkey brain (and plausibly also in the human brain) which are dual purpose, visuomotor, responding both to the perception of an action and to the planning and execution of the action. Three recent studies seem relevant for this paper. Fadiga et. al. 2002 provide evidence that listening to speech specifically modulates the excitability of neurons for the tongue muscles;the listener understands the speaker when neural centres for his/her own articulatory gestures are activated (as proposed on the basis of the 1967 psycho-physical experiments by Liberman and his Haskins colleagues). Fadiga et al. have demonstrated, using transcranial magnetic stimulation (TMS) , that, as a subject listens to spoken words, an increase in motor-evoked potentials can be recorded from the listener’s tongue muscles when the words involve strong tongue movements; they have shown for the first time that listening produces activation for specific phonemes in speech motor centres. The observation/execution matching system (mirror system) is thought to be the physiological expression of a brain mechanism making possible understanding of the actions of others; the agent and the observer share the same motor action repertoire. Similarly, the phoneme recognition mechanism could be involved in phonetically ‘understanding’ others’ speech because the speaker and the listener share the same articulatory motor repertoire.
Another recent study may throw light on the paradox of echolalia in autistic children coupled with a frequent imitative deficit otherwise. Decety et al. 2002 exploring the neural mechanisms involved in reciprocal imitation found that the left brain area involved in producing imitation had the homologous area in the right brain more activated in perceiving one’s actions imitated by another; a possible neural basis for distinguishing internally produced actions from those generated by others.
A study by Kohler et al. 2002 “Hearing sounds, understanding actions” found that, in research with monkeys, many object-related actions can be recognised by their sound. Multifunctional neurons in the premotor cortex discharge when the animal performs a specific actione, when it hears the sound associated with the action and also when it observes the action. If humans also have these mirror neurons (as seems likely) the existence of audiovisual neurons suggests how the meaning of actions could be linked to hearing spoken language.The neural vocabulary contains not only schemas on how an action should be executed but also action ideas and may be a key to gestural communication and the evolution of spoken language.
Mirror neurons, motor primitives and motor imagery research fit extremely well with the motor theory of language developed a number of years ago (Allott 1991). The motor theory is that there is a direct relation between the functioning of speech and motor control generally, with language depending on pre-existing motor primitives coupled with the operation of motor equivalence. The motor system not only executes actions but also internally represents them in terms of ‘motor ideas’. PET and fMRI scanning demonstrates that bodily action is preceded by a mental picturing of the proposed action. A perceptually-organised pattern is transduced into a motor program and executed by the changes in posture, changes in limb positions which constitute human action. The concept of the motor program has evolved from earlier understanding of motor action in terms of ‘schemas’ and motor ‘patterns’. At the highest level the motor program may be better described as an ‘action program’; at the lowest level there are motor sub-routines, motor elements combined in chains and in combination. Many aspects of motor behaviour, and particularly expressive motor behaviour, are found in new-born infants; the neural connections to support the behaviour must have been established before birth. Low-level primitives have been found in the neural circuits of the spinal cord of the frog organised into a number of distinct functional modules. These force fields are computational primitives used by the CNS for generating a rich grammar of motor behaviors in analogy with language primitives used to generate unlimited sentences. The speech and skeletomuscular systems share common neural control modes. Kinematic patterns for movements of the tongue dorsum are similar to those of voluntary flexion-extension movements about the elbow. In research on articulation the model used for speech was exactly the model used for controlling arm movements, with the articulators of the vocal tract simply substituted for those of the arm, the articulatory gesturesacting as phonological primitives. The motor control of the tongue is a section of the total system of motor control for all bodily movement. Tongue movements can be explained in terms of a small number of independent muscle groups, each corresponding to an elementary or ‘primitive’ movement, making use of a relatively small number of invariant muscle synergies. The human tongue is not likely to be less complicated in its muscles and neural control than the incredible complexity of the muscle and neural control of the dog’s tongue on which the research has necessarily been conducted. If motor primitives are essential to avoid the degrees of freedom combinatorial explosion for movements of the arm or leg, it seems inevitable that they are even more acutely required for the overwhelming complexity of tongue musculature, tongue activity and neural control. Bernstein described the concept of motor equivalence (1967): the motor image or motor idea floats free of the particular limb or particular muscle/joint assembly usually employed to execute the motor program. “I can write the letter A with my hand, with my foot, or even with my mouth; I could even make an A by walking on the beach” [Berthoz 1997]; the neural basis for this equivalence, writing with the finger and the big toe, has been verified using fMRI. Motor equivalence is demonstrated most remarkably in the relation between speech and gesture. Motor equivalence can operate from speech to gesture or from gesture to speech. It also seems likely that it can operate between other modalities and speech - or more precisely between speech and perception and between perception and motor action because visual perception is also a motor activity with its own control system and motor primitives. Though there is no space to set out the practical evidence in this papere, what is found to be the case for the control of bodily movements is paralleled by the speech and language production system: Individual speech sounds are motor primitives Words formed from primitive speech-sound elements are motor programs Movements of the arm are the motor equivalents of the speech primitives Before we produce a sentence there is a motor image of the sentence A sentence is a high-level motor program or action plan.
Language and and imitation are closely related by the intermediation of motor control. Behaviour- based robotics is well-placed to take advantage of neuroscience research focusing on these two major ascpects of human functioning.
Aitchison, J. The Articulate Mammal: An introduction to psycholinguistics. London:Unwin Hyman 1989.
Allott, R. The Origin of Language: The General Problem. In Studies in Language Origins I. Eds. J. Wind, E.G. Pulleyblank, É. de Grolier and B.H. Bichakjian. Amsterdam: Benjamins 1989.
Allott, R. . The Motor Theory of Language. In Studies in Language Origins II. Eds. W. von Raffler-Engel, J. Wind and A. Jonker. Amsterdam: Benjamins 1991.
Allott, R. Autism and the Motor Theory of Language. LOS Selwyn College, Cambridge. September 1992. httm://www.percepp.demon.co.uk/autism.htm
Amit, R. and M.J. Mataric. Parametric Primitives for Motor Representation and control. Int.Conf. on Robotics and Automation Washington D.C. May 11-15 2002 2002.
Berthoz, A. Le Sens du Mouvement. Paris: Editions Odile Jacob 1997.
Billard, A. and K. Dautenhahn. Experiments in learning by imitation - grounding and use of communication in robotic agents. Adaptive Behavior, 7(3/4):415-438 1999.
Billard, A. and M.J. Mataric. Learning human arm movements by imitation: evaluation of a biologically inspired connectionist architecture. Robotics and Autonomous Systems 941 2001 1-16 2001.
Bloomfield, L. Language. Allen and Unwin. 1970 Reprint [1933]
Breazeal, C. and B. Scassellati. Challenges in Building Robots that imitate people. MIT AI Laboratory http://www.ai.mit.edu/projects/humanoid-robotics-group/cog/cog-publications/MITPress-Im itation.pdf.
Chomsky, N. Language and Problems of Knowledge. MIT Press Cambridge MA 1988.
Dautenhahn, K. Evolvability, Culture and the Primate Social Brain. http://homepages.feis.herts.ac.uk/~nehaniv/al7ev/dautenhahn/dautenhahn.html
Deacon, T. W. Symbolic Species: The Co-evolution of Language and the Brain. New York: W.W. Norton 1997.
Decety, J. ed. Cerveau, Perception et Action Psychologie Française. Tome 45 no. 4. 2000.
Decety, J., T. Chaminade, J. Grèzes and A.N. Meltzoff. A PET exploration of the neural mechanismsinvolved in reciprocal imitation. Neuroimage 15 (1) 265-272 2002.
Fadiga, L., G. Craighero, G. Buccino and G. Rizzolatti. Speech listening specifically modulates the excitability of tongue muscles: a TMS study. European Journal of Neuroscience Vol.15 pp. 399-402 2002.
Fadiga L, L. Fogassi, G. Pavesi and G. Rizzolatti. Motor facilitation during action observation: a magnetic stimulation study. Neurophysiol 73(6):2608-11 1995.
Gopnik, A., A. Meltzoff and P. Kuhl. The Scientist in the Crib: Minds, Brains, and How Children Learn. New York: William Morrow 1999.
Gregory, R.L. Language Learning: Word Meanings. In The Oxford Companion to the Mind, 421 1987.
Grèzes, J., J. Decety and P. Fonlupt 319-322 Equivalence fonctionelle: mythe cognitif où realitè neurologique? In J. Decety, ed. Cerveau, Perception et Action Psychologie Française. Tome 45 no. 4. 2000.
Helmuth, L. From the Mouths (and Hands) of Babes. Science Vol 293 1758-9 2001.
Hobson, P. The Cradle of Thought: Exploring the Origins of Thinking. London: Macmillan 2002.
Hurley, S.L. Consciousness in Action. Cambridge MA: Harvard UP 1998.
Iacoboni, M., R.P. Woods, M. Brass, H. Bekkering, J.C. Mazziotta and G. Rizzolatti. Cortical Mechanisms of Human Imitation. Science 286 2526-2528 1999.
Kohler, E., C. Keysers, M.A. Umilta, L. Fogassi, V. Gallese and G. Rizzolatti. Hearing Sounds Understanding Actions: Action Representation in Mirror Neurons. Science 297 846-8 2002.
Lenneberg, E.H. Biological Foundations of Language. New York: John Wiley 1967.
Lieberman, P. The biology and evolution of language. Cambridge, Mass.: Harvard UP.1984.
Lieberman, P. Uniquely Human: Speech, Thought, and Selfless Behavior. Harvard University Press 1991.
Mataric, M.J. Learning by imitation. http://www-robotics.usc.edu/~maja/imitation.html
Mataric, M.J. Sensory-Motor Primitives as a Basis for Imitation: Linking Perception to Action and Biology to Robotics. In C. Nehaniv, K. Dautenhahn eds. Imitation in Animals and Artifacts. MIT Press 2002.
Mataric, M. J. Getting Humanoids to move and imitate. IEEE Intelligent Systems 1094 18-24 2000.
Mataric, M.J., O. C. Jenkins, A. Fod, and V. Zordan. Control and Imitation in Humanoids. AAAI Fall Symposium on Simulating Human Agents, North Falmouth, MA, Nov 3-5, 2000.
Meltzoff, A.N. Elements of a developmental theory of imitation. In Meltzoff, A. N. and W. Prinz. eds. The Imitative Mind, 19-141 Cambridge UP 2002.
Meltzoff, A.N. and M.K. Moore. Newborn infants imitate adult facial gestures. Child Dev. 54, 702- 709. 1983.
Meltzoff, A.N. and M.K. Moore. Explaining facial imitation: A Theoretical Model. Early Development and parenting. 1997.
Pinker, S. The Language Instinct: The New Science of Language and Mind. Penguin. 1995.
Pinker, S. Words and Rules: The Ingredients of Language. London: Phoenix 2001.
Rizzolatti, G. and M. Arbib. Language within our grasp. Trends in Neuroscience 21 188-194 1998.
Rizzolatti, G, L. Fadiga, V. Gallese and L. Fogassi. Premotor cortex and the recognition of motor actions. Cognitive Brain Research 3(2) 131-141 1996.
Studdert Kennedy, M. Development of the speech perceptuo-motor system. In Precursors of early speech. ed. by B. Lindblom and R. Zetterstrom, 206-217. New York: Stockton Press. 1986.
Trevarthen, C. How control of movement develops. In Human Motor Actions: Bernstein Reassessed. ed. by H. Whiting, 223-261. Amsterdam: North-Holland. 1984.
Trevarthen, C. Infant semiosis. In Origins of Semiosis: Sign Evolution in Nature and Culture. ed. W. Nöth, 219-252. Mouton de Gruyter 1994.
Weber, S., M.J. Mataric, O.C. Jenkins. Experiments in imitation using perceptuo-motor primitives. Agents 136-137 2000.
Williams, J.H.G., A. Whiten, T. Suddendorf and D.I. Perrett. Imitation, mirror neurons and autism. Neuroscience and Biobehavioral Reviews 25 287-295 2001.