creators_name: van der Velde, Frank
type: techreport
datestamp: 2003-10-09
lastmod: 2011-03-11 08:55:21
metadata_visibility: show
title: Neural blackboard architectures of combinatorial structures in cognition
ispublished: unpub
subjects: neuro-mod
subjects: cog-psy
subjects: comp-sci-neural-nets
full_text_status: public
keywords: Binding, blackboard architectures, combinatorial structure, compositionality, language, neurocognition, sentence complexity, sentence structure, working memory, variables, vision
abstract: Human cognition is unique in the way in which it relies on combinatorial (or compositional) structures. Language provides ample evidence for the existence of combinatorial structures, but they can also be found in visual cognition. To understand the neural basis of human cognition, it is therefore essential to understand how combinatorial structures can be instantiated in neural terms. In his recent book on the foundations of language, Jackendoff described four fundamental problems for a neural instantiation of combinatorial structures: the massiveness of the binding problem, the problem of 2, the problem of variables and the transformation of combinatorial structures from working memory to long-term memory. This paper aims to show that these problems can be solved by means of neural ‘blackboard’ architectures. For this purpose, a neural blackboard architecture for sentence structure is presented. In this architecture, neural structures that encode for words are temporarily bound in a manner that preserves the structure of the sentence. It is shown that the architecture solves the four problems presented by Jackendoff. The ability of the architecture to instantiate sentence structures is illustrated with examples of sentence complexity observed in human language performance. Similarities exist between the architecture for sentence structure and blackboard architectures for combinatorial structures in visual cognition, derived from the structure of the visual cortex. These architectures are briefly discussed, together with an example of a combinatorial structure in which the blackboard architectures for language and vision are combined. In this way, the architecture for language is grounded in perception.  
date: 2003
date_type: published
institution: Leiden University
department: Cognitive Psychology
refereed: FALSE
referencetext: Amit, D. (1989) Modeling brain function. Cambridge University Press. 
Amit, D. (1995) The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behavioral and Brain Sciences 18:617-657.
Anderson, J. R. (1983) The architecture of cognition. Cambridge, MA: Harvard University Press. 
Barsalou, L. W. (1999) Perceptual symbol systems. Behavioral and Brain Sciences 22:577-660.  
Barsalou, L. W., Simmons, W. K., Barbey, A. K. & Wilson, C. D. (2003) Grounding conceptual knowledge in modality-specific systems. Trends in Cognitive Sciences 7:84-91. 
Batista, A. P., Buneo, C. A. Snyder, L. H. & Andersen, R. A. (1999) Reach plans in eye-centered coordinates. Science 285:257-260. 
Bechtel, W. & Abrahamsen, A. (2002) Connectionism and the mind. Malden, MA: Blackwell. 
Bierwisch, M. (1999) Words in the brain are not just labeled concepts. Behavioral and Brain Sciences 22:280-282.
Bloom, P. (2000) How children learn the meanings of words. Cambridge, MA: MIT Press.
Bulier J. (2001) Feedback connections and conscious vision. Trends in Cognitive Sciences 5:369-370.
Calvin, W. H. & Bickerton, D. (2000) Lingua ex machina: Reconciling Darwin and Chomsky with the human brain. Cambridge, MA: MIT Press.
Caplan, D. (1992) Language: Structure, processing and disorders. Cambridge, MA: MIT Press.
Chelazzi, L., Miller, E. K., Duncan, J. & Desimone, R. (1993) A neural basis for visual search in inferior temporal cortex. Nature 363:345-347.
Chomsky, N. (2000) New horizons in the study of language and mind. Cambridge University Press. 
Christiansen, M. H., & Chater, N. (1999) Toward a connectionist model of recursion in human linguistic performance. Cognitive Science 23:157-205.
Christiansen, M.H. & Chater, N. (2001) Connectionist psycholinguistics: capturing the empirical data. Trends in Cognitive Sciences 5:82-88.
Churchland, P. M. (1995) The engine of reason, the seat of the soul. Cambridge, MA: MIT Press. 
De Kamps, M., van der Velde, F. (2001) Using a recurrent network to bind form, color and position into a unified percept. Neurocomputing 38-40: 523-528.
Dennett, D. C. (1991) Consciousness explained. London: Allen Lane.
Desimone, R. & Duncan, J. (1995) Neural mechanisms of selective visual attention. Annual Review of Neuroscience 18:193-222.
D'Esposito, M. (2001) Functional neuroimaging of working memory. In: Handbook of functional neuroimaging of cognition, eds. R. Cabeza and A. Kingstone. Cambridge, MA: MIT Press.  
Duncan, J. (2001) An adaptive coding model of neural function in prefrontal cortex. Nature Reviews Neuroscience 2:820-829.
Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000) Neurocomputational models of working memory. Nature Neuroscience 3 (sup):1184-1191.
Elman, J. L. (1991) Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning 7:195-225.
Farah, M., Humphries, G. W., Rodman, H. R. (1999) Object and face recognition. In: Fundamental neuroscience, eds. M. J. Zigmond, F. E. Bloom, S. C. Landis, J. L. Roberts and L.R. Squire. San Diego: Academic Press.
Felleman, D. J. & van Essen, D. C. (1991) Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex 1:1-47.
Fodor, J. A. & McLaughlin, B. P. (1990) Connectionism and the problem of systematicity - Why Smolensky’s solution doesn’t work. Cognition 35:183-204. 
Fodor, J. A., & Pylyshyn, Z. W. (1988) Connectionism and cognitive architecture: A critical analysis. Cognition 28:3-71. 
Fuster, J. M. (1973) Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. Journal of Neurophysiology 36:61-78.
Fuster, J. M. (1995) Memory in the cerebral cortex. Cambridge, MA: MIT Press.
Fuster, J. M. (2001) The prefrontal cortex-An update: Time is of the essence. Neuron 30:319-333.
Fuster, J. M., Bauer, R. H., & Jervey, J. P. (1985) Functional interactions between inferotemporal and prefrontal cortex in a cognitive task. Brain Research 330:299-307.
Gibson, E. (1998) Linguistic complexity: Locality of syntactic dependencies. Cognition 68:1-76. 
Gonchar, Y. Burkhalter, A. (1999) Connectivity of GABAergic calretinin-immunoreactive neurons in rat primary visual cortex. Cerebral Cortex 9:683-696.
Grodzinsky, Y. (2000) The neurology of syntax: Language use without Broca’s area. Behavioral and Brain Sciences 23:1-71.
Harnad, S. (1991) The symbol grounding problem. In: Emergent computation: Self-organizing, collective, and cooperative phenomena in natural and artificial computing networks, ed. S. Forrest. Cambridge, MA: MIT Press. 
Hawkins, J. A. (1994) A performance theory of order and constituency. Cambridge University Press.
Hebb, D. O. (1949) The organisation of behaviour. New York: Wiley.
Hubel, D. (1995) Eye, brain, and vision. New York: Scientific American Library. 
Jackendoff, R. (2002) Foundations of language. Oxford: Oxford University Press.
Jackendoff, R. (in press) Précis to Foundations of language. Behavioral and Brain Sciences. 
Lewis, R. L. (1999) Accounting for the fine structure of syntactic working memory: Similarity-based interference as a unifying principle. Behavioral and Brain Sciences 22:105-106.
Livingstone, M.S. & Hubel, D. H. (1988) Segregation of form, colour, movement and depth: Anatomy, physiology, and perception, Science 240:740-749.
Luck, S. J. & Vogel, E. K. (1997) The capacity of visual working memory for features and conjunctions. Nature 390:279-281.
Marcus, G. F. (2001) The algebraic mind. Cambridge, MA: MIT Press.
McNaughton, B. L. & Nadel, L. (1990) Hebb-Marr networks and the neurobiological representation of action in space. In: Neuroscience and connectionist theory, ed. M. A. Gluck & D. E. Rumelhart. Hillsdale, NJ: Erlbaum. 
Miikkulainen, R. (1996) Subsymbolic case-role analysis of sentences with embedded clauses. Cognitive Science 20:47-73.
Motter, B. C. (1994) Neural correlates of attentive selection for color or lumimance in extrastriate area V4. Journal of Neuroscience 14:2178-2189. 
Nadel, L. & Moscovitch, M. (2001) The hippocampal complex and long-term memory revisited. Trends in Cognitive Sciences 5:228-230.
Newell, A. (1990) Unified theories of cognition. Cambridge, MA: Harvard University Press.
Oram, M. W. Perrett, D. I. (1994) Modeling visual recognition from neurobiological contraints. Neural Networks 7:945-972.
O'Reilly, R. C. & Rudy, J. W. (2001) Conjunctive representations in learning and memory: Principles of cortical and hippocampal function. Psychological Review 108:311-345.
Palmer-Brown, D., Tepper, J. A. & Powell, H. M. (2002) Connectionist natural language parsing. Trends in Cognitive Sciences 6:437-442.
Piattelli-Palmarini, M. (2002) The barest essentials. Nature 416:129.
Pinker, S. (1994). The language Instinct. London: Penguin.
Pinker, S. (1998) How the mind works. London: Penguin.
Port, R. F. & van Gelder, T. eds. (1995) Mind as motion: Explorations in the dynamics of cognition. Cambridge, MA: MIT Press.
Posner, M.I., & Petersen, S. E. (1990) The attention system of the human brain. Annual Review of Neuroscience 13:25-42.
Prabhakaran, V. Narayanan, K. Zhao, Z. & Gabrieli. J. D. E. (2000) Integration of diverse information in working memory within the frontal lobe. Nature Neuroscience 3:85-90.
Pullum, G. K. & Scholz, B. C. (2001) More than words. Nature 413:367.
Pulvermüller, F. (1999) Words in the brain's language. Behavioral and Brain Sciences 22:253-336.
Pulvermüller, F. (2000) Syntactic circuits: How does the brain create serial order in sentences?. Brain and Language 71:194-199.
Pulvermüller, F., Harle, M & Hummel, F. (2001) Walking or talking? Behavioral and neurophysiological correlates of action verb processing. Brain and Language 78:143-168.
Pylyshyn, Z. (2002) Mental imagery: In search of a theory. Behavioral and Brain Sciences 25:157-238.
Raffone & Wolters (2001) A cortical mechanism for binding in visual working memory. Journal of Cognitive Neuroscience 13:766-785. 
Rainer, G. Assad, W. F. & Miller, E. K. (1998) Memory fields of neurons in the primate prefrontal cortex. Proceedings of the National Academy of Science USA 95: 15008-15013.
Rao, S. C., Rainer, G. & Miller, E. K. (1997) Integration of what and where in the primate prefrontal cortex. Science 276:821-824.
Rolls, E. T. & Treves, A. (1998) Neural networks and Brain Function. Oxford University Press. 
Ruchlin, D. S., Grafman, J., Caneron, K. & Berndt, R. S. (in press) Working memory retention systems: A state of activated long-term memory. Brain and Behavioral Sciences. 
Sag, I. A. & Wasow, T. (1999) Syntactic theory. Stanford, CA: CSLI Publications.
Selfridge, O. G. (1959) Pandemonium: A paradigm for learning. In: Proceedings of the symposium on the mechanisation of thought processes eds. D. V. Blake and A. M. Uttley. London: HM Stationary Office. 
Shastri L. & Ajjanagadde, V. (1993) From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences 16:417-494.
Van der Heijden, A. H. C., Kurvink, A. G., de Lange, L., de Leeuw, F. & van der Geest, J. N. (1996) Attending to color with proper fixation. Perception & Psychophysic 58:1224-1237. 
Van der Heijden A. H. C. & van der Velde, F. (1999) Cognition for selection. Visual Cognition 6:83-87. 
Van der Velde, F. (1995) Symbol manipulation with neural networks: Production of a context-free language using a modifiable working memory. Connection Science 7:247-280. 
Van der Velde, F. (1997) On the use of computation in modelling behavior. Network: Computation in Neural Systems 8:1-32.
Van der Velde, F. (1999) A spy to spy on a spy: From type to token representation with cell assemblies. Behavioral and Brain Sciences 22:306-307.
Van der Velde, F. & de Kamps, M. (2001) From knowing what to knowing where: Modeling object-based attention with feedback disinhibition of activation. Journal of Cognitive Neuroscience 13:479-491.
Van der Velde, F. & de Kamps, M. (2002) Synchrony in the eye of the beholder: An analysis of the role of neural synchronization in cognitive processes. Brain and Mind 3:291-312.
Van der Velde, F. & de Kamps, M. (2003a) A model of visual working memory in PFC. Neurocomputing 52-54: 419-424.
Van der Velde, F. & de Kamps, M. (2003b) Neural assembly binding in linguistic representation. In: European Symposium on Artificial Neural Networks. ed. M. Verleysen. Evere: d-side publications.  
Van der Velde, F. & de Kamps, M. & Van der Voort van der Kleij, G. T. (in press) CLAM: Closed-loop attention model for visual search. Neurocomputing.
Van der Velde, F., Van der Voort van der Kleij, G. T. & de Kamps, M. (2003) Lack of combinatorial productivity in language processing with Simple Recurrent Networks. Technical Report. Leiden University. 
Van der Voort van der Kleij, G. T., de Kamps, M. & van der Velde, F. (2003) A neural model of binding and capacity in visual working memory. Lecture Notes in Computer Science. 2714:771-778.
Van Gelder, T. (1990) Compositionality: A connectionist variation on a classical theme. Cognitive Science, 14:355-384.
Van Valin, R. D. Jr. (2001) An introduction to syntax. Cambridge: Cambridge University Press. 
Von der Malsburg, C. (1987) Synaptic plasticity as basis of brain organization. In: The Neural and Molecular Bases of Learning, eds. J. P. Changeux and M. Konishi. Chichester: John Wiley.
Webelhuth, G. ed. (1995) Government and binding theory and the minimalist program. Oxford: Blackwell.
Wilson, F.A.W., Ó Scalaidhe, S. P., & Goldman-Rakic, P. S. (1993) Dissociation of object and spatial processing domains in primate prefrontal cortex. Science  260:1955-1958.
Yakovlev, V. Fusi, S., Berman, E. & Zohary, E. (1998) Inter-trial neuronal activity in inferior temporal cortex: A putative vehicle to generate long-term associations. Nature Neuroscience 1:310-317. 

citation:   van der Velde, Dr. Frank  (2003) Neural blackboard architectures of combinatorial structures in cognition.  [Departmental Technical Report]    (Unpublished)  
document_url: http://cogprints.org/3196/1/neuralarchs.pdf