Natural Language Processing with Modular Neural Networks and Distributed Lexicon

Miikkulainen, Risto and Dyer, Michael G. (1991) Natural Language Processing with Modular Neural Networks and Distributed Lexicon. [Journal (Paginated)]

Full text available as:

[img] Postscript


An approach to connectionist natural language processing is proposed, which is based on hierarchically organized modular Parallel Distributed Processing (PDP) networks and a central lexicon of distributed input/output representations. The modules communicate using these representations, which are global and publicly available in the system. The representations are developed automatically by all networks while they are learning their processing tasks. The resulting representations reflect the regularities in the subtasks, which facilitates robust processing in the face of noise and damage, supports improved generalization, and provides expectations about possible contexts. The lexicon can be extended by cloning new instances of the items, that is, by generating a number of items with known processing properties and distinct identities. This technique combinatorially increases the processing power of the system. The recurrent FGREP module, together with a central lexicon, is used as a basic building block in modeling higher level natural language tasks. A single module is used to form case-role representations of sentences from word-by-word sequential natural language input. A hierarchical organization of four recurrent FGREP modules (the DISPAR system) is trained to produce fully expanded paraphrases of script-based stories, where unmentioned events and role fillers are inferred.

Item Type:Journal (Paginated)
Keywords:modular networks, parsing, generation, word representations
Subjects:Computer Science > Language
Computer Science > Neural Nets
Linguistics > Computational Linguistics
ID Code:1896
Deposited By: Miikkulainen, Risto
Deposited On:18 Nov 2001
Last Modified:11 Mar 2011 08:54


Repository Staff Only: item control page