Abstract

In this talk I will present a multidimensional approach to the interpretation and generation of utterances in multimodal dialogue, and argue that such an approach has numerous benefits.

What is mean by a ‘multidimensional’ approach is that participation in a dialogue is viewed as being involved in multiple, parallel activities such as advancing a dialogue-mediated task or activity; giving and providing communicative feedback; taking turns; monitoring the use of time; editing one’s own and one’s partner’s speech, and so on. These various types of activity are called ‘dimensions’, and the activities that are performed in each dimension are called ‘dialogue acts.’ Dialogue acts are defined semantically in terms of update operations on the participants’ mental states; in particular on their beliefs, goals, and obligations. This view has been worked out in considerable detail in the framework known as Dynamic Interpretation Theory (DIT).

One of the results of analyzing dialogues in the DIT framework has been the development of a fine-grained 10-dimensional taxonomy of dialogue act types, called the DIT ++ taxonomy (http://dit.uvt.nl), which has been applied in human and computer recognition, annotation, and generation of dialogue acts.

I will outline the framework of Dynamic Interpretation Theory and the DIT ++ taxonomy, and show the benefits of this framework for (1) the segmentation of (spoken and multimodal) dialogue behavior into meaningful units; (2) the analysis and explanation of the phenomenon that dialogue utterances often have more than one communicative function; (3) the annotation of spoken and multimodal dialogue with semantic information; (4) the complex semantics of discourse connectives; and (5) the generation of deliberately multifunctional utterances by a dialogue system.