title: Conjunctive Visual and Auditory Development via Real-Time Dialogue
creator: Zhang, Yilu
creator: Weng, Juyang
subject: Machine Learning
subject: Robotics
description: Human developmental learning is capable of dealing with the dynamic visual world, speech-based dialogue, and their complex real-time association. However, the architecture that realizes this for robotic cognitive development has not been reported in the past. This paper takes up this challenge. The proposed architecture does not require a strict coupling between visual and auditory stimuli. Two major operations contribute to the “abstraction” process: multiscale temporal priming and high-dimensional numeric abstraction through internal responses with reduced variance. As a basic principle of developmental learning, the programmer does not know the nature of the world events at the time of programming and, thus, hand-designed task-specific representation is not possible. We successfully tested the architecture on the SAIL robot under an unprecedented challenging multimodal interaction mode: use real-time speech dialogue as a teaching source for simultaneous and incremental visual learning and language acquisition, while the robot is viewing a dynamic world that contains a rotating object to which the dialogue is referring.
publisher: Lund University Cognitive Studies
contributor: Prince, Christopher G.
contributor: Berthouze, Luc
contributor: Kozima, Hideki
contributor: Bullock, Daniel
contributor: Stojanov, Georgi
contributor: Balkenius, Christian
date: 2003
type: Conference Paper
type: PeerReviewed
format: application/pdf
identifier: http://cogprints.org/3342/1/Zhang.pdf
identifier:   Zhang, Yilu and Weng, Juyang  (2003) Conjunctive Visual and Auditory Development via Real-Time Dialogue.  [Conference Paper]     
relation: http://cogprints.org/3342/