Cogprints

Integrated 2-D Optical Flow Sensor

Stocker, Dr. Alan (2004) Integrated 2-D Optical Flow Sensor. [Preprint]

Full text available as:

[img]
Preview
 PDF
1479Kb

Abstract

I present a new focal-plane analog VLSI sensor that estimates optical flow in two visual dimensions. The chip significantly improves previous approaches both with respect to the applied model of optical flow estimation as well as the actual hardware implementation. Its distributed computational architecture consists of an array of locally connected motion units that collectively solve for the unique optimal optical flow estimate. The novel gradient-based motion model assumes visual motion to be translational, smooth and biased. The model guarantees that the estimation problem is computationally well-posed regardless of the visual input. Model parameters can be globally adjusted, leading to a rich output behavior. Varying the smoothness strength, for example, can provide a continuous spectrum of motion estimates, ranging from normal to global optical flow. Unlike approaches that rely on the explicit matching of brightness edges in space or time, the applied gradient-based model assures spatiotemporal continuity on visual information. The non-linear coupling of the individual motion units improves the resulting optical flow estimate because it reduces spatial smoothing across large velocity differences. Extended measurements of a 30x30 array prototype sensor under real-world conditions demonstrate the validity of the model and the robustness and functionality of the implementation.

Item Type:Preprint
Additional Information:this is a preprint/technical report that will be published soon in a journal. check http://www.cns.nyu.edu/~alan/publications/publications.htm for upcoming versions.
Keywords:visual motion perception, 2-D optical flow, constraint optimization, gradient descent, aVLSI, analog network, collective computation, neuromorphic, feedback, nonlinear smoothing, perceptual prior
Subjects:Neuroscience > Computational Neuroscience
Computer Science > Machine Vision
Computer Science > Robotics
ID Code:3377
Deposited By: Stocker, Dr. Alan
Deposited On:13 Jan 2004
Last Modified:11 Mar 2011 08:55

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

[1] C. Mead, “Neuromorphic electronic systems,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1629–1636, October 1990. [2] P. Nesi, F. Innocenti, and P. Pezzati, “Retimac: Real-time motion analysis chip,” IEEE Trans. on Circuits and Systems - 2: Analog and Digital Signal Processing, vol. 45, no. 3, pp. 361–375, March 1998.

[3] E. C. Hildreth, The Measurment of Visual Motion. MIT Press, 1983.

[4] D. Murray and B. Buxton, “Scene segmentation from visual motion using global optimization,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 9, no. 2, pp. 220–228, March 1987.

[5] J. Hutchinson, C. Koch, J. Luo, and C. Mead, “Computing motion using analog and binary resistive networks,” Computer, vol. 21, pp. 52–64, March 1988.

[6] T. Sejnowski and S. Nowlan, “A model of visual motion processing in area MT of primates,” in The Cognitive Neuroscience, MIT, Ed. MIT, 1995, pp. 437–449.

[7] M. Chang, A. Tekalp, and M. Sezan, “Simultaneous motion estimation and segmentation,” IEEE Trans. on Image Processing, vol. 9, no. 6, pp. 1326–1333, September 1997. [8] J. Shi and J. Malik, “Motion segmentation and tracking using normalized cuts,” in Intl. Conference on Computer Vision, January 1998.

[9] A. Stocker, “An improved 2-D optical flow sensor for motion segmentation,” in IEEE Intl. Symposium on Circutis and Systems, vol. 2, 2002, pp. 332–335.

[10] T. Horiuchi, J. P. Lazzaro, A. Moore, and C. Koch, “A delay-line based motion detection chip,” in Advances in Neural Information Processing Systems 3, R. Lippman, J. Moody, and D. Touretzky, Eds., 1991, vol. 3, pp. 406–412. [11] R. Etienne-Cummings, S. Fernando, N. Takahashi, V. Shotonov, J. Van der Spiegel, and P. M ̈uller, “A new temporal domain optical flow measurement technique for focal plane VLSI implementation,” in Computer Architectures for Machine Perception, December 1993, pp. 241–250.

[12] R. Sarpeshkar, W. Bair, and C. Koch, “Visual motion computation in analog VLSI using pulses,” in Advances in Neural Information Processing Systems 5, 1993, pp. 781–788. [13] J. Kramer, “Compact integrated motion sensor with three-pixel interaction,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 455–460, April 1996.

[14] C. Higgins and C. Koch, “Analog CMOS velocity sensors,” Procs. of Electronic Imaging SPIE, vol. 3019, Februar 1997. [15] T. Horiuchi, B. Bishofberger, and C. Koch, “An analog VLSI saccadic eye movement system,” in Advances in Neural Information Processing Systems 6, J. Cowan, G. Tesauro, and J. Alspector, Eds. Morgan Kaufmann, 1994, pp. 582–589.

[16] R. Deutschmann and C. Koch, “An analog VLSI velocity sensor using the gradient method,” in Procs. IEEE Intl. Symposium on Circuits and Systems. IEEE, 1998, pp. 649–652. [17] A. Andreou and K. Strohbehn, “Analog VLSI implementation of the Hassenstein-ReichardtPoggio models for vision computation.” Procs. of the 1990 Intl. Conference on Systems, Man and Cybernetics, 1990.

[18] R. Benson and T. Delbruck, “Direction-selective silicon retina that uses null inhibition,” in Neural Information Processing Systems 4, D. Touretzky, Ed. MIT Press, 1992, pp. 756–763.

[19] T. Delbruck, “Silicon retina with correlation-based velocity-tuned pixels,” IEEE Trans. on Neural Networks, vol. 4, no. 3, pp. 529–541, May 1993.

[20] R. Harrison and C. Koch, “An analog VLSI model of the fly elementary motion detector,” in Advances in Neural Information Processing Systems 10, M. Kearns and S. Solla, Eds. MIT Press, 1998, pp. 880–886.

[21] M. Ohtani, T. Asai, H. Yonezu, and N. Ohshima, “Analog velocity sensing circuits based on bio-inspired correlation neural networks,” in Microelectronics for Neural, Fuzzy and BioInspired Systems. Granada, Spain, 1999, pp. 366–373.

[22] S.-C. Liu, “A neuromorphic aVLSI model of global motion processing in the fly,” IEEE Trans. on Circuits and Systems 2, vol. 47, no. 12, pp. 1458–1467, 2000.

[23] C. Higgins, R. Deutschmann, and C. Koch, “Pulse-based 2D motion sensors,” IEEE Trans. on Circuits and Systems 2: Analog and Digital Signal Processing, vol. 46, no. 6, pp. 677–687, June 1999.

[24] R. Deutschmann and C. Koch, “Compact real-time 2D gradient-based analog VLSI motion sensor,” in Intl. Conference on Advanced Focal Plane Arrays and Electronic Cameras, 1998.

[25] H.-C. Jiang and C.-Y. Wu, “A 2-D velocity- and direction-selective sensor with BJT-based silicon retina and temporal zero-crossing detector,” IEEE Journal of Solid-State Circuits, vol. 34, no. 2, pp. 241–247, February 1999.

[26] J. Kramer, R. Sarpeshkar, and C. Koch, “Pulse-based analog VLSI velocity sensors,” IEEE Trans. on Circuits and Systems 2, vol. 44, no. 2, pp. 86–101, February 1997.

[27] R. Etienne-Cummings, J. Van der Spiegel, and P. Mueller, “A focal plane visual motion measurement sensor,” Trans. on Circuits and Systems I, vol. 44, no. 1, pp. 55–66, January 1997.

[28] J. Tanner and C. Mead, “An integrated analog optical motion sensor,” in VLSI Signal Processing, 2, S.-Y. Kung, R. Owen, and G. Nash, Eds. IEEE Press, 1986, p. 59 ff.

[29] A. Moore and C. Koch, “A multiplication based analog motion detection chip,” in SPIE Visual Information Processing: From Neurons to Chips, B. Mathur and C. Koch, Eds., vol. 1473. SPIE, 1991, pp. 66–75.

[30] A. Stocker and R. Douglas, “Computation of smooth optical flow in a feedback connected analog network,” in Advances in Neural Information Processing Systems 11, M. Kearns, S. Solla, and D. Cohn, Eds. Cambridge, MA: MIT Press, 1999, pp. 706–712.

[31] M.-H. Lei and T.-D. Chiueh, “An analog motion field detection chip for image segmentation,” IEEE Trans. on circuits and systems for video technology, vol. 12, no. 5, pp. 299–308, May 2002.

[32] J. Limb and J. Murphy, “Estimating the velocity of moving images in television signals,” Computer Graphics Image Processing, vol. 4, pp. 311–327, 1975.

[33] B. Horn and B. Schunck, “Determining optical flow,” Artificial Intelligence, vol. 17, pp. 185– 203, 1981.

[34] M. Snyder, “On the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 13, no. 11, pp. 1105–1114, November 1991.

[35] A. Yuille and N. Grzywacz, “A mathematical analysis of the motion coherence theory,” Intl. Journal of Computer Vision, vol. 3, pp. 155–175, 1989.

[36] E. Simoncelli, E. Adelson, and D. Heeger, “Probability distributions of optical flow,” in IEEE Conference on Computer Vision and Pattern Recognition. IEEE, june 1991, pp. 310–313.

[37] Y. Weiss, E. Simoncelli, and E. Adelson, “Motion illusions as optimal percept,” Nature Neuroscience, vol. 5, no. 6, pp. 598–604, June 2002.

[38] J. Harris, C. Koch, E. Staats, and J. Luo, “Analog hardware for detecting discontinuities in early vision,” Intl. Journal of Computer Vision, vol. 4, pp. 211–223, 1990. [39] J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Procs. National Academic Sciences U.S.A., vol. 79, pp. 2554–2558, April 1982.

[40] T. Delbruck and C. Mead, “Analog VLSI phototransduction by continuous-time, adaptive, logarithmic photoreceptor cirucuits,” Caltech Computation and Neural Systems Program, Tech. Rep. 30, 1994.

[41] S.-C. Liu, “Silicon retina with adaptive filtering properties,” in Advances in Neural Information Processing Systems 10. MIT Press, November 1998, pp. 712–718.

[42] C. Mead, Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley, 1989.

[43] E. Simoncelli, “Design of multi-dimensional derivative filters,” in Frist IEEE Conference on Image Processing Austin. IEEE, November 1994.

[44] B. Gilbert, “A precise four-quadrant multiplier with subnanosecond response,” IEEE Journal of Solid-State Circuits, vol. 3, no. 4, pp. 363–373, December 1968.

[45] A. Stocker, “Constraint optimization networks for visual motion perception - analysis and synthesis,” Ph.d. Thesis no. 14360, Swiss Federal Institute of Technology ETHZ, Z ̈urich, Switzerland, March 2002, www.cns.nyu.edu/∼alan/publications/publications.htm.

[46] P. Gray and R. Meyer, Analysis and Design of Analog Integrated Circuits, 3rd ed., 93, Ed. New York: Wiley and sons, 1993.

[47] J. Harris, C. Koch, and J. Luo, “A two-dimensional analog VLSI circuit for detecting discontinuities in early vision,” Science, vol. 248, pp. 1209–1211, June 1990.

[48] T. Delbruck, “Investigations of analog VLSI visual transduction and motion processing,” Ph.D. dissertation, Department of Computational and Neural Systems, California Institute of Technology, Pasadena, CA, 1993.

[49] D. Rumelhart, G. Hinton, and J. McClelland, Parallel distributed processing: Explorations in the microstructure of cognition, D. Rumelhart, J. McClelland, and the PDP Research Group, Eds. MIT Press, 1986, vol. 1.

[50] R. Rao and D. Ballard, “The visual cortex as a hierachical predictor,” Dept. of Computer Science, University of Rochester, Tech. Rep. 96.4, 1996.

[51] C. Pack and R. Born, “Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain,” Nature, vol. 409, pp. 1040–1042, February 2001.

[52] J. Heinzle and A. Stocker, “Classifying patterns of visual motion - a neuromorphic approach,” in Advances in Neural Information Processing Systems 15, S. T. S. Becker and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2003, pp. 1123–1130.

[53] V. Becanovic, G. Indiveri, H.-U. Kobialka, Pl ̈oger, and A. A. Stocker, Mechatronics and Machine Vision 2002: Current Practice, ser. Robotics. Research Studies Press, 2002, ch. Silicon Retina Sensing guided by Omni-directional Vision, pp. 13–21.

[54] S. Treue and J. Maunsell, “Attentional modulation of visual motion processing in cortical areas MT and MST,” Nature, vol. 382, pp. 539–541, August 1996.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.