Abstract

ABSTRACT : In this talk I will present a formal semantic analysis of iconic and deictic gesture. The model describes the incomplete meaning of gesture that’s derivable from its form, and the pragmatic reasoning that yields a more specific interpretation. Our formalism builds on established models of discourse interpretation to capture key insights from the descriptive literature on gesture. For example, synchronous speech and iconic gesture express a single thought, but while the form of iconic gesture is an important clue to its interpretation, the content of an unconventionalised gesture can be resolved only by linking it to its context. Specifically, we utilise discourse structure to model how context augments the content of gesture that’s revealed by its form, and dynamic semantics to model constraints on co-reference across speech and gesture, and across sequences of gesture.

This work is joint with Matthew Stone (Rutgers University)