Cogprints

Neural Mechanisms for Information Compression by Multiple Alignment, Unification and Search

Wolff, Dr J G (2003) Neural Mechanisms for Information Compression by Multiple Alignment, Unification and Search. [Preprint]

Full text available as:

[img]
Preview
 PDF
676Kb

Abstract

This article describes how an abstract framework for perception and cognition may be realised in terms of neural mechanisms and neural processing. This framework — called information compression by multiple alignment, unification and search (ICMAUS) — has been developed in previous research as a generalized model of any system for processing information, either natural or artificial. It has a range of applications including the analysis and production of natural language, unsupervised inductive learning, recognition of objects and patterns, probabilistic reasoning, and others. The proposals in this article may be seen as an extension and development of Hebb’s (1949) concept of a ‘cell assembly’. The article describes how the concept of ‘pattern’ in the ICMAUS framework may be mapped onto a version of the cell assembly concept and the way in which neural mechanisms may achieve the effect of ‘multiple alignment’ in the ICMAUS framework. By contrast with the Hebbian concept of a cell assembly, it is proposed here that any one neuron can belong in one assembly and only one assembly. A key feature of present proposals, which is not part of the Hebbian concept, is that any cell assembly may contain ‘references’ or ‘codes’ that serve to identify one or more other cell assemblies. This mechanism allows information to be stored in a compressed form, it provides a robust mechanism by which assemblies may be connected to form hierarchies and other kinds of structure, it means that assemblies can express abstract concepts, and it provides solutions to some of the other problems associated with cell assemblies. Drawing on insights derived from the ICMAUS framework, the article also describes how learning may be achieved with neural mechanisms. This concept of learning is significantly different from the Hebbian concept and appears to provide a better account of what we know about human learning.

Item Type:Preprint
Keywords:cell assembly, information compression, multiple alignment
Subjects:Neuroscience > Computational Neuroscience
Computer Science > Artificial Intelligence
Neuroscience > Behavioral Neuroscience
ID Code:2824
Deposited By: Wolff, J G
Deposited On:12 Mar 2003
Last Modified:11 Mar 2011 08:55

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network, 3:213–251.

Attneave, F. (1954). Some informational aspects of visual perception. PsychologicalReview, 61:183–193.

Baddeley, R., Hancock, P., and Foldiak, P., editors (2000). Information Theory and the Brain. Cambridge University Press, Cambridge.

Barlow, H. B. (1959). Sensory mechanisms, the reduction of redundancy, and intelligence. In HMSO, editor, The Mechanisation of Thought Processes, pages 535–559. Her

Majesty’s Stationery Office, London.

Barlow, H. B. (1961). The coding of sensory messages. In Thorpe, W. H. and Zangwill, O. L., editors, Current Problems in Animal Behaviour, pages 331–360. Cambridge

University Press, Cambridge. Chapter XIII.

Barlow, H. B. (1969). Trigger features, adaptation and economy of impulses. In Leibovic, K. N., editor, Information Processes in the Nervous System, pages 209–230. Springer, New York.

Barlow, H. B. (1972). Single units and sensation: a neuron doctrine for perceptual psychology? Perception, 1:371–394.

Barlow, H. B. (1982). David Hubel and Torsten Wiesel: their contribution towards understanding the primary visual cortex. Trends in Neuroscience, 5(5):145–152.

Barlow, H. B. (1983). Intelligence, guesswork, language. Nature, 304:207–209.

Barlow, H. B. (1997). The knowledge used in vision and where it comes from. Philosophical Transactions of the Royal Society London B, 352:1141–1147.

Barlow, H. B. (2001a). The exploitation of regularities in the environment by the brain. Behavioural and Brain Sciences, 24(4):602–607 and 748–749.

Barlow, H. B. (2001b). Redundancy reduction revisited. Network: Computation in Neural Systems, 12:241–253.

Chater, N. (1996). Reconciling simplicity and likelihood principles in perceptual organisation. Psychological Review, 103(3):566–581.

Chater, N. (1999). The search for simplicity: a fundamental cognitive principle? Quarterly Journal of Experimental Psychology, 52 A(2):273–302.

Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. John Wiley, New York.

Gardner-Medwin, A. R. and Barlow, H. B. (2001). The limits of counting accuracy in distributed neural representations. Neural Computation, 13:477–504.

Garner, W. R., editor (1974). The Processing of Information and Structure. Lawrence Erlbaum, Hillsdale, NJ.

Gold, M. (1967). Language identification in the limit. Information and Control, 10:447–474.

Hebb, D. O. (1949). The Organization of Behaviour. John Wiley & Sons, New York.

Hebb, D. O. (1958). A Textbook of Psychology. W. B. Saunders Company, Philadelphia.

Hubel, D. H. (2000). Exploration of the primary visual cortex, 1955–78. In Gazzaniga, M. S., editor, Cognitive Neuroscience: a Reader, pages 58–80. Blackwell, Oxford.

Huyck, C. R. (2001). Cell assemblies as an intermediate level model of cognition. In Wermter, S., Austin, J., and Willshaw, D., editors, Emergent Neural Computational

Architectures Based on Neuroscience, pages 383–397. Springer, Heidelberg.

Li, M. and Vitanyi, P. (1997). An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, New York.

Marr, D. (1982). Vision: a Computational Investigation into the Human Representation and Processing of Visual Information. W.H. Freeman, San Francisco.

Milner, P. M. (1957). The cell assembly: mark ii. Psychological Review, 64(4):242–252.

Milner, P. M. (1996). Neural representations: some old problems revisited. Journal of Cognitive Neuroscience, 8(1):69–77.

Milner, P. M. (1999). The Autonomous Brain. Lawrence Erlbaum Associates, Mahwah, New Jersey.

Nicholls, J. G., Martin, A. R., Wallace, B. G., and Fuchs, P. A. (2001). From Neuron to Brain. Sinauer Associates, Sunderland, Mass.

Oldfield, R. C. (1954). Memory mechanisms and the theory of schemata. British Journal of Psychology, 45:14–23.

Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco.

Pulvermuller, F. (1999). Words in the brain’s language. Behavioural and Brain Sciences, 22(2):253–336.

Pulvermuller, F. (2002). A brain perspective on language mechanisms: from discrete neuronal ensembles to serial order. Progress in Neurobiology, 67:85–111.

Ratliff, F. and Hartline, H. K. (1959). The response of limulus optic nerve fibres to patterns of illumination on the receptor mosaic. Journal of General Physiology, 42:1241–1255.

Ratliff, F., Hartline, H. K., and Miller, W. H. (1963). Spatial and temporal aspects of retinal inhibitory interaction. Journal of the Optical Society of America, 53:110–120.

Rissanen, J. (1978). Modelling by the shortest data description. Automatica-J, IFAC, 14:465–471.

Sakitt, B. and Barlow, H. B. (1982). A model for the economical encoding of the visual image in cerebral cortex. Biological Cybernetics, 43:97–108.

Sakurai, Y. (1998). Cell-assembly coding in several memory processes. Neurobiology of Learning and Memory, 70:212–225.

Sankoff, D. and Kruskall, J. B. (1983). Time Warps, String Edits, and Macromolecules: the Theory and Practice of Sequence Comparisons. Addison-Wesley, Reading, MA.

Sholl, D. A. (1956). The Organisation of the Cerebral Cortex. Wiley, New York.

Shors, T. J. and Gould, E. (2001). xxx. Nature, xx:xx.

Snyder, A. W. and Barlow, H. B. (1988). Revealing the artist’s touch. Nature, 331:117–118.

Solomonoff, R. J. (1964). A formal theory of inductive inference. parts I and II. Information and Control, 7:1–22 and 224–254.

Solomonoff, R. J. (1997). The discovery of algorithmic probability. Journal of Computer and System Sciences, 55(1):73–88.

Storer, J. A. (1988). Data Compression: Methods and Theory. Computer Science Press, Rockville, Maryland.

Thompson, M. (2000). Coding third-order image structure. In Baddeley et al. (2000), pages 62–78.

von Bekesy, G. (1967). Sensory Inhibition. Princeton University Press, Princeton, NJ.

Wallace, C. S. and Boulton, D. M. (1968). An information measure for classification. Computer Journal, 11(2):185–195.

Watanabe, S. (1972). Pattern recognition as information compression. In Watanabe, S., editor, Frontiers of Pattern Recognition. Academic Press, New York.

Williams, R. W. and Herrup, K. (2001). The control of neuron number. The Annual Review of Neuroscience, 11:423–453.

Wolff, J. G. (1975). An algorithm for the segmentation of an artificial language analogue. British Journal of Psychology, 66:79–90.

Wolff, J. G. (1977). The discovery of segments in natural language. British Journal of Psychology, 68:97–106.

Wolff, J. G. (1980). Language acquisition and the discovery of phrase structure. Language & Speech, 23:255–269.

Wolff, J. G. (1982). Language acquisition, data compression and generalization. Language & Communication, 2:57–89.

Wolff, J. G. (1988). Learning syntax and meanings through optimization and distributional analysis. In Levy, Y., Schlesinger, I. M., and Braine, M. D. S., editors, Categories and Processes in Language Acquisition, pages 179–215. Lawrence Erlbaum, Hillsdale, NJ.

Wolff, J. G. (1990). Simplicity and power—some unifying ideas in computing. Computer Journal, 33(6):518–534.

Wolff, J. G. (1993). Computing, cognition and information compression. AI Communications, 6(2):107–127.

Wolff, J. G. (1994). A scaleable technique for best-match retrieval of sequential information using metrics-guided search. Journal of Information Science, 20(1):16–28.

Wolff, J. G. (1998). Probabilistic reasoning as information compression by multiple alignment, unification and search. Technical report, School of Informatics, University of

Wales at Bangor. Copy: www.cognitionresearch.org.uk/papers/pr/pr.htm.

Wolff, J. G. (1999a). ‘Computing’ as information compression by multiple alignment, unification and search. Journal of Universal Computer Science, 5(11):777–815.

Wolff, J. G. (1999b). Probabilistic reasoning as information compression by multiple alignment, unification and search: an introduction and overview. Journal of Universal Computer Science, 5(7):418–462.

Wolff, J. G. (2000). Syntax, parsing and production of natural language in a framework of information compression by multiple alignment, unification and search. Journal of

Universal Computer Science, 6(8):781–829.

Wolff, J. G. (2001a). Information compression by multiple alignment, unification and search as a framework for human-like reasoning. Logic Journal of the IGPL, 9(1):205–222. First published in the Proceedings of the International Conference on Formal and Applied Practical Reasoning (FAPR 2000), September 2000, ISSN 1469–4166.

Wolff, J. G. (2001c). Information compression by multiple alignment, unification and search as a framework for ‘intelligent’ computing. In Proceedings of the AISB’01

Symposium on Nonconscious Intelligence: from Natural to Artificial, pages 422–428. The

Society for the Study of Artificial Intelligence and the Simulation of Behaviour (AISB).

ISBN 1 902956 21 3. The Symposium formed part of the AISB’01 Convention, York.

Wolff, J. G. (2001b). Information compression by multiple alignment, unification and search as a model of non-conscious intelligence. In Proceedings of the AISB’01

Symposium on Nonconscious Intelligence: from Natural to Artificial, pages 72–81. The Society for the Study of Artificial Intelligence and the Simulation of Behaviour (AISB). ISBN 1 902956 21 3. The Symposium formed part of the AISB’01 Convention, York.

Wolff, J. G. (2002a). Mathematics and logic as information compression by multiple alignment, unification and search. Technical report, CognitionResearch.org.uk. Copy:

www.cognitionresearch.org.uk/papers/cml/cml.htm.

Wolff, J. G. (2002b). Unsupervised learning in a framework of information compression by

multiple alignment, unification and search. Technical report, CognitionResearch.org.uk.

Wolff, J. G. (to appear). Information compression by multiple alignment, unification and

search as a unifying principle in computing and cognition. Artificial Intelligence Review.

Zipf, G. K. (1935). The Psycho-Biology of Language. Houghton Mi²in, Boston.

Zipf, G. K. (1949). Human Behaviour and the Principle of Least E®ort. Hafner, New York.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.