Applying semantic concepts to the media assigment problem in multi-media communication.

Stenning, Keith and Inder, Robert (1995) Applying semantic concepts to the media assigment problem in multi-media communication. [Book Chapter]

Full text available as:

[img] Postscript


Our long term goal is an understanding of human communication in terms which would provide the basis for rational design. The kernel would be a theory of the cognitive consequences of allocating the same information to different media and modalities, based on the user's information processing characterised in computational terms. Our theory of the cognitive consequences of media/modality allocation starts from an analysis of differences in logical expressiveness of graphical and linguistic representations (Stenning \& Oberlander (1994, 1995)). This semantic approach requires conceptualisations of {\it medium} and {\it modality} that can be related to representation systems. We propose that media are the physical/perceptual aspects of representations; modalities are classes of interpretation function which map media onto meanings. These interpretations of the terms contrast with existing HCI usage. The curtailments of abstraction observed in graphics arise from interactions between medium and modality. Graphical modalities are distinguished from sentential languages by the nature of their interpretation functions. A hierarchy of expressiveness of interpretations of graphics is defined, and compared with interpretations of sentential languages. Using the expressiveness of representations to predict their cognitive properties also requires reference to the availability of constraints of their interpretation to users. Further contrasts between graphics and language emerge in the availability of their constraints. Three example domains of graphical representations are analysed from this perspective---matrix graphics; logic diagrams; and semantic networks. Some empirical evidence of the usability of these notations is reviewed as evidence that the proposed conceptualisation offers powerful generalisations.

Item Type:Book Chapter
Keywords:diagrammatic reasoning, cognitive complexity, computationalcomplexity, linguistic and diagrammatic semantics, multimodality, multimedia
Subjects:Psychology > Cognitive Psychology
Computer Science > Language
Linguistics > Semantics
Philosophy > Philosophy of Language
Philosophy > Logic
ID Code:822
Deposited By: Stenning, Keith
Deposited On:05 Aug 1999
Last Modified:11 Mar 2011 08:54


Repository Staff Only: item control page