Learning Low Dimensional Representations of Visual Objects With Extensive Use of Prior Knowledge

Abstract

Learning to recognize visual objects from examples requires the ability to find meaningful patterns in spaces of very high dimensionality. We present a method for dimensionality reduction which effectively biases the learning system by combining multiple constraints via an extensive use of class labels. The use of multiple class labels steers the resulting low-dimensional representation to become invariant to those directions of variation in the input space that are irrelevant to classification; this is done merely by making class labels independent of these directions. We also show that prior knowledge of the proper dimensionality of the target representation can be imposed by training a multiple-layer bottleneck network. A series of computational experiments involving parameterized fractal images and real human faces indicate that the low-dimensional representation extracted by our method leads to improved generalization in the learned tasks, and is likely to preserve the topology of the original space.