Cogprints

Evolving modular architectures for neural networks

Di Ferdinando, Andrea and Calabretta, Raffaele and Parisi, Domenico (2001) Evolving modular architectures for neural networks. [Conference Paper] (In Press)

Full text available as:

[img] PDF
31Kb

Abstract

Neural networks that learn the What and Where task perform better if they possess a modular architecture for separately processing the identity and spatial location of objects. In previous simulations the modular architecture either was hardwired or it developed during an individual's life based on a preference for short connections given a set of hardwired unit locations. We present two sets of simulations in which the network architecture is genetically inherited and it evolves in a population of neural networks in two different conditions: (1) both the architecture and the connection weights evolve; (2) the network architecture is inherited and it evolves but the connection weights are learned during life. The best results are obtained in condition (2). Condition (1) gives unsatisfactory results because (a) adapted sets of weights can suddenly become maladaptive if the architecture changes, (b) evolution fails to properly assign computational resources (hidden units) to the two tasks, (c) genetic linkage between sets of weights for different modules can result in a favourable mutation in one set of weights being accompanied by an unfavourable mutation in another set of weights.

Item Type:Conference Paper
Keywords:evolution of modularity, neural networks, genetic algorithms, what and where system
Subjects:Biology > Evolution
Psychology > Cognitive Psychology
Neuroscience > Computational Neuroscience
Computer Science > Neural Nets
ID Code:1298
Deposited By: Calabretta, Raffaele
Deposited On:10 Feb 2001
Last Modified:11 Mar 2011 08:54

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

1. Belew, R. K., McInerney, J., & Schraudolph, N. (1991). Evolving networks: using the genetic algorithm with connectionist learning. In C. G. Langton, C. Taylor, J. D. Farmer, & S. Rasmussen (eds), Artificial Life II. Addison-Wesley, Reading, MA.

2. Belew, R. K. & Mitchell, M. (1996). Adaptive Individuals in Evolving Populations. Addison-Wesley, Reading, MA.

3. Calabretta, R., Nolfi, S., Parisi, D. & Wagner, G. P. (2000). Duplication of modules facilitates the evolution of functional specialization. Artificial Life 6:69-84.

4. Cangelosi A., Parisi D. & Nolfi S. (1994). Cell division and migration in a 'genotype' for neural networks. Network 5:497-515.

5. Elman, J. L., Bates, E. A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D. & Plunkett, K. (1996). Rethinking innateness. A connectionist perspective on development. The MIT Press, Cambridge, MA.

6. Floreano, D. & Urzelai, J. (2000). Evolutionary robots with on-line self-organization and behavioral fitness. Neural Networks 13:431-443.

7. Jacobs, R. A., Jordan, M. I. & Barto, A. G. (1991). Task decomposition through competition in a modular connectionist architecture: The what and where vision tasks. Cognitive Science 15:219-250.

8. Jacobs, R. A. & Jordan, M. I. (1992). Computational consequences of a bias toward short connections. Journal of Cognitive Neuroscience 4:323-335.

9. Kolen J. F. & Pollack, J. B. (1990). Back-propagation is sensitive to initial conditions. Complex Systems 4:269-280.

10. Murre, J. M. J. (1992). Learning and categorization in modular neural networks. Harvester, New York, NY.

11. Plaut D. C. & Hinton, G. E. (1987). Learning sets of filters using back-propagation. Computer Speech and Language 2:35-61.

12. Reed, R. D. & Marks II, R. J. (1999). Neural Smithing. Supervised Learning in Feedforward Artificial Neural Networks. The MIT Press, Cambridge, MA.

13. Rueckl, J. G., Cave, K. R. & Kosslyn, S. M. (1989). Why are “what” and “where” processed by separate cortical visual systems? A computational investigation. Journal of Cognitive Neuroscience 1:171-186.

14. Ungerleider, L. G. & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale & R. J. W. Mansfield (Eds.), The Analysis of Visual Behavior. The MIT Press, Cambridge, MA.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.