Continuous Computation and the Emergence of the Discrete

Abstract

This paper argues that the foundation of expertise and skillful behavior is knowledge represented as a structured continuum as opposed to a discrete structure. The reason is that continuous knowledge representation permits more flexible behavior and avoids failures due to brittleness. As a consequence it is crucial that we understand the principles of continuous representation so that they can be applied in artificial intelligence and neuroscience. To this end I have compared and contrasted continuous (``analog'') and discrete (``digital'') computation, and have suggested a theoretical framework (simulacra) for continuous representations. This theoretical framework was used to characterize several kinds of invariance that occur in natural and artificial systems, including as special cases such psychologically significant invariances as symmetry groups and other algebraic invariances involved in object perception. Several ways of quantifying degree of invariance are noted since quantification is a necessary prerequisite to understanding the emergence of algebraic structure in perceptual and cognitive processes. The paper reviews several simple mechanisms, based on correlational learning, that lead to the emergence of invariances. Finally, it describes metric correlation, a generalization of the familiar correlation and convolution operations, which accomplishes transform invariant pattern recognition.