ANTLIMA -- A Listener Model with Mental Images

Abstract

AI research concerning the connection between seeing and speaking mainly employs what is often called reference semantics. Applying this approach to the situation of a radio sports reporter, we have to coordinate the demand of referentially anchoring an utterance dealing with the visually perceived, and the demand for coherence of an utterance as part of a verbal interaction with somebody not situated in the same perceptual context. In consequence, we are led to the conception of a speaker anticipating the listeners' understanding by means of mental images which replace the percepts being described, and thus provide the referents for the audience. We present a system realizing this type of partner modeling, emphasizing mainly the reconstruction of the referents, i.e., of a mental image. Starting from the thesis that the audience expects the speaker to mean the most typical case of the described class of events or situations with respect to the communicated context, we explain a mechanism for representing and using typicality distributions of static spatial relations which is related to Herskovits' analytical framework. Extended to restrictions of speed and temporal duration, this mechanism also allows us to construct dynamic mental images corresponding to the referents of objective sports reports.