Cogprints

A neural blackboard architecture of sentence structure

van der Velde, dr. Frank and de Kamps, dr. Marc (2003) A neural blackboard architecture of sentence structure. [Departmental Technical Report] (Unpublished)

Full text available as:

[img]
Preview
 PDF
771Kb

Abstract

We present a neural architecture for sentence representation. Sentences are represented in terms of word representations as constituents. A word representation consists of a neural assembly distributed over the brain. Sentence representation does not result from associations between neural word assemblies. Instead, word assemblies are embedded in a neural architecture, in which the structural (thematic) relations between words can be represented. Arbitrary thematic relations between arguments and verbs can be represented. Arguments can consist of nouns and phrases, as in sentences with relative clauses. A number of sentences can be stored simultaneously in this architecture. We simulate how probe questions about thematic relations can be answered. We discuss how differences in sentence complexity, such as the difference between subject-extracted versus object-extracted relative clauses and the difference between right-branching versus center-embedded structures, can be related to the underlying neural dynamics of the model. Finally, we illustrate how memory capacity for sentence representation can be related to the nature of reverberating neural activity, which is used to store information temporarily in this architecture.

Item Type:Departmental Technical Report
Keywords:Neural sentence structure, neural assemblies, neural dynamics, compositional representation
Subjects:Neuroscience > Neurolinguistics
Neuroscience > Neural Modelling
Psychology > Cognitive Psychology
ID Code:3271
Deposited By: van der Velde, Dr. Frank
Deposited On:14 Nov 2003
Last Modified:11 Mar 2011 08:55

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Abeles, M. (1991). Corticonics. Cambridge: Cambridge University Press.

Amit, D. J. (1989). Modeling brain function. Cambridge: Cambridge University Press.

Amit, D. J. (1995). The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behavioral and Brain Sciences, 18, 617-657.

Amit, D. J. & Brunel, N. (1997). Global spontaneous activity and local structured (learned) delay activity in cortex. Cerebral Cortex, 7, 237-252.

Bates, E. & Goodman, J. C. (1997). On the inseparability of grammar and the lexicon: Evidence from acquisition, aphasia and real-time processing. Language and Cognitive processes, 12, 507-584.

Bloom, P. (2000). How children learn the meanings of words. Cambridge, MA: MIT Press.

Brunel, N. (2000). Persistent activity and single-cell frequency-current curve in a cortical network model. Network: Computation in Neural Systems, 11, 261-280.

Calvin, W. H. (1995). Cortical columns, modules, and Hebbian cell assemblies. In M. A. Arbib, G. Adelman and P. H. Arbib (Eds.) The handbook of brain theory and neural networks, (pp. 269-272). Cambridge, MA: MIT Press.

Calvin W. H. & Bickerton, D. (2000). Lingua ex machina: Reconciling Darwin and Chomsky with the human brain. Cambridge, MA: MIT Press.

Caplan, D., Baker, C. & Dehaut, F. (1985). Syntactic determinants of sentence comprehension in aphasia. Cognition, 21, 117-175.

Caplan, D. & Futter, C. (1986). Assignment of thematic roles by an agrammatic aphasic patient. Brain and Language, 27, 117-135.

Caramazza, A. & Zurif, E. (1976). Dissociation of algorithmic and heuristic processes in sentence comprehension: Evidence from aphasia. Brain and Language, 3, 572-582.

Chomsky, N. (2000). New horizons in the study of language and mind. Cambridge: Cambridge University Press.

Christiansen, M. H., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157-205.

Compte, A., Brunel, N., Goldman-Rakic, P. S. & Wang, X. J. Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. Cerebral Cortex, 10, 910-923.

Elman, J. L. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195-225.

Fodor, J. A. & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: a critical analysis. Cognition, 50, 151-170.

Fuster, J. M. (1973). Unit activity in prefrontal cortex during delayed-response performance: Neuronal correlates of transient memory. Journal of Neurophysiology, 36, 61-78.

Gerstner, W. (1995). Time structure of the activity in neural-network models. Physical Review E, 51, 738-758.

Gerstner, W. (2000). Population dynamics of spiking neurons: fast transients, asynchronous states, and locking. Neural Computation, 12, 43-89.

Gibson, E. (1998). Linguistic complexity: Locality of syntactic dependencies. Cognition, 68, 1-76.

Gonchar, Y. & Burkhalter, A. (1999). Connectivity of GABAergic calretinin-immunoreactive neurons in rat primary visual cortex. Cerebral Cortex, 9, 683-696.

Grodzinsky, Y. & Finkel, L. (1998). The neurology of empty categories: Aphasics' failure to detect ungrammaticality. Journal of Cognitive Neuroscience, 10(2), 281-292.

Grodzinsky, Y. (2000). The neurology of syntax: Language use without Broca's area. Behavioral and Brain Sciences, 23, 1-17.

Grossberg, S. (2000) The complementary brain: unifying brain dynamics and modularity. Trends in Cognitive Sciences, 4, 233-246.

Hebb, D. O. (1949). The Organisation of Behaviour. New York : Wiley.

Hickok, G., Zurif, E. & Canseco-Gonzalez, E. (1993). Structural description of agrammatic comprehension. Brain and Language, 45, 371-395.

Itty, L. & Koch, C. (2001) Computational modelling of visual attention. Nature Reviews Neuroscience. 2, 194-203.

Jackendoff, R. (1999). Parallel constraint-based generative theories of language. Trends in Cognitive Sciences, 3, 393-400.

Just, M. A., Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122-149.

Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F. & Thulborn, K. R. (1996). Brain activation modulated by sentence comprehension. Science, 274, 114-116.

Keller, T. A., Carpenter, P. A. & Just. M. A. (2001). The neural basis of sentence comprehension: a fMRI examination of syntactic and lexical processing. Cerebral Cortex, 11, 223-237.

Koulakov, A. A. (2001). Properties of synaptic transmission and the global stability of delayed activity states. Network: Computation in Neural Systems, 12, 47-74.

Lewis, R. (1996). A theory of grammatical but unacceptable embeddings. Journal of Psycholinguistic Research, 25, 93-116.

MacDonald, M. C., Perlmutter, N. J. & Seidenberg, M. S. (1994). The lexical nature of syntactic ambiguity resolution. Psychological Review, 101, 676-703.

McClelland, J. L. & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents of sentences. In J. L. McClelland and D. E. Rumelhart (eds.) Parallel Distributed Processing Vol 2 (pp 272-325), Cambridge, MA: MIT Press.

Miller, E. K. (2000). The prefrontal cortex and cognitive control. Nature Reviews Neuroscience, 1, 59-65.

Mountcastle, V. B. (1998). The Cerebral Cortex. Cambridge, MA: Harvard University Press.

Miikkulainen, R. (1996). Subsymbolic case-role analysis of sentences with embedded clauses. Cognitive Science, 20, 47-73.

Pinker, S. (1989). Learnability and cognition. Cambridge, MA: MIT Press.

Pinker, S. (1994). The language instinct. London: Penguin.

Pinker, S. (1998). How the mind works. London: Penguin

Pulvermüller, F. (1999). Words in the brain's language. Behavioral and Brain Sciences, 22, 253-336.

Pulvermüller, F. (2001). Brain reflections of words and their meaning. Trends in Cognitive Sciences, 5, 517-524.

Sag, I. A. & Wasow, T. (1999). Syntactic theory: A formal introduction. Stanford: CSLI Lecture Notes.

Sougné. J. (1998). Connectionism and the problem of multiple instantiation. Trends in Cognitive Sciences, 2, 183-189.

Stabler, E. P. (1994). The finite connectivity of linguistic structures. In: C. Clifton, L. Frazier and K. Rayner (Eds.), Perspectives on sentence processing. Hillsdale, NJ: Erlbaum.

Stromswold, K., Caplan, D., Alpert, N. & Rauch, S. (1996). Localization of syntactic comprehension by positron emission tomography. Brain and Language, 52, 452-473.

Usher, M. & Niebur, E. (1996). Modeling the temporal dynamics of IT neurons in visual search: A mechanism for top-down selective attention. Journal of Cognitive Neuroscience, 8, 311-327.

Van der Velde, F. (1995). Symbol manipulation with neural networks: Production of a context-free language using a modifiable working memory. Connection Science, 7, 247-280.

Van der Velde, F. (1997). On the use of computation in modelling behavior. Network: Computation in Neural Systems, 8, 1-32.

Van der Velde, F. (2001). The study of brain function: Methodological considerations. In:. Amit, D. & Parisi, G. (Eds), Frontiers of life, Vol III, part two: Building blocks for intelligence systems. San Diego: Academic Press.

Van der Velde, F. & de Kamps, M. (2001). From knowing what to knowing where: Modeling object-based attention with feedback disinhibition of activation. Journal of Cognitive Neuroscience, 13, (4), 479-491.

Van der Velde, F., Van der Voort van der Kleij, G. T. & de Kamps, M. (subm). Lack of combinatorial productivity in language processing with simple recurrent networks.

Van Valin, R. D. (2001). An introduction to syntax. Cambridge: Cambridge University press.

Wang, X-J. (2001). Synaptic reverberation underlying mnemonic persistent activity. Trends in Neurosciences, 24, 455-463.

Wilson, H. R. & Cowan, J. D. (1972). Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal, 12, 1-14.

Vosse, Th. & Kempen, G. (2000). Syntactic structure assembly in human parsing: A computational model based on competitive inhibition and a lexicalist grammar. Cognition, 75, 105-143.

Webelhuth, G. (ed.), (1995). Government and binding theory and the minimalist program. Cambridge, MA: Blackwell.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.