The Effect of Aural and Visual Presentation Formats on Categorical and Dimensional Judgements of Emotion for Sung and Spoken Expressive Performances

The purpose of this study was to investigate the effects of visual information on the perception of emotion in three contexts: a short spoken phrase, an analogous short melody, and a longer melody with greater complexity of pitch and rhythm. Participants without substantial formal music training were assigned to either an audio-only, visual-only, or audiovisual presentation mode; all observed musicians and actors who sang melodies or spoke phrases intending to communicate happiness, sadness, or anger. Participants rated these performances for positive and negative valence, energy arousal, and tension arousal. They were also asked to select the discrete emotion they perceived, and to rate how certain they were about this selection.