Abstract

The ability to display and recognise emotions is an important aspect of social interaction between humans. We monitor each other’s facial expressions, vocal nuances and body posture and gestures, and use them to make inferences about other people’s mental states. Our understanding of mental states shapes the decisions that we make, governs how we communicate with others, and affects our performance. People express these social signals even when we are interacting with machines, but computer interfaces currently ignore them. In effect, computers are autistic. Recent advances in psychology have greatly improved our understanding of the role of affect in communication, perception, decision-making, attention and memory. At the same time, advances in technology mean that it is becoming possible for machines to sense, analyse and express emotions. Computer systems with emotional awareness can analyse a person’s facial expressions, tone of voice and body posture and gestures. They infer a person’s underlying mental state, such as whether he or she is agreeing or disagreeing, interested or bored, thinking or confused. Similar techniques endow humanoid robots with the ability to display the same signals. The techniques have applications in areas such as monitoring cognitive load in command and control operators, guiding on-line teaching systems and enhancing the sense of presence in teleconference systems.