Abstract

In this talk I will present an automatic inference system of expressions of emotions and mental states from non-verbal speech which outperforms human accuracy. The inference is based on a new set of vocal features including music derived features that have not been used before in this context. I will describe how an arbitrary set of expressions can be used to automatically map a set of several hundred lexical definitions of mental states using the inference machine, and find the vocal correlates of a lexically oriented topology of mental states. I will also show how findings from one language (English) can be used to automatically infer expressions in another language (Hebrew), demonstrating expressions in human-computer interaction (HCI) which involves decision-making, the complex behaviour of expressions in time, and the verification through multi-modal and context analysis.