Graphical Communication in Human-Computer Interaction

A theory of the cognitive impact of choice of modalities in communication between people and machines will be developed, especially the choice and combination of language, graphics and animation: ie, the modality allocation problem. The theory is to serve as the basis for generalising interface taxonomy, design and evaluation. It will take a modality-independent characterisation of information and then explore the cognitive effects of translating the same information into different media and combinations of media. Hence semantic theory will guide interface design which undergoes psychological evaluation to study the impact of closely comparable information presented in different modality combinations.
Research is being carried out in order to develop a theory of the cognitive impact of modality choice in communication between people and machines, especially the choice and combination of language, graphics and animation.

The study has focused on foundational issues in the semantics and pragmatics of graphical representations, and on gathering data on: graphically based interactions between humans; interface design practice; and the effects on learning of exposure to graphics.