Now Reading
Sounds, shapes, speech and body movements convey emotion through one shared property

Sounds, shapes, speech and body movements convey emotion through one shared property

People communicate their emotions using their voice, face, and movement, as well as through abstract forms such as art, architecture, and music. The structure of these expressions is frequently intuitively related to their meaning: flowery curlicues are used in romantic poetry, while a spiky script is used in death metal band logos.

A Dartmouth study published in the Proceedings of the Royal Society B explain how this code is used across sounds, shapes, speech and human body movements, providing a strong multi-sensory signal that can be used to efficiently estimate an agent’s level of emotional arousal.

“Our study set out to better understand how we express and read emotional arousal, which is fundamental to our core emotional state. We wanted to see if there is a low-level mechanism that allows us to decode emotional arousal information from the movements and sounds that people and animals make,” says lead author Beau Sievers, a postdoctoral student of psychology at Harvard University, who was a graduate student in the department of psychological and brain sciences at Dartmouth at the time of the study.

The spectral centroid is a multi-sensory metric of spikyness. The findings explain why Zen gardens and brutalist architecture have extremely distinct emotional impacts, as well as why clouds and lullabies appear to go together despite the fact that one is seen and the other is heard: we match them based on the spectral centroid.

The researchers performed five mini-tests to see if the spectral centroid is utilized to convey and comprehend emotional arousal. Some of the experiments required people to make judgments about the emotional arousal of forms, sounds, and motions. The researchers investigated whether the stimulus’s spectral centroid might be utilized to predict participants’ emotional arousal judgments. Highlights from three of the mini-studies are as follows:

A pointy shape here represents anger, while a blobby shape represents sadness
Characteristic angry and sad drawings from study 3, after smoothing and corner detection. Corners are marked with red ‘+’ signs. Among the study’s participants, angry drawings had a mean of 23.3 corners, while sad drawings had a mean of 6.6 corners. The image is credited to Beau Sievers et al.

• The writers utilized computer software to generate hundreds of shapes and noises at random. Participants were instructed to gaze at shapes, listen to noises, and rate their emotional arousal levels. According to the study, forms and sounds with a high spectral centroid were linked with high-arousal emotions (angry, enthusiastic), whereas shapes and sounds with a low spectral centroid were connected with low-arousal emotions (sad, peaceful).

• Participants were instructed to sketch angry, sad, joyful, or tranquil forms. The researchers then calculated the spectral centroids of the drawings by counting the number of corners. The findings indicated that furious and enthusiastic forms had an average of 17 to 24 corners, whereas sad and serene shapes had an average of 7 to 9 corners. With over 80% accuracy, the spectral centroid may be utilized to predict the emotional arousal of forms.

Figure 2.
Shape pairs used in tasks 1 and 2, based on Köhler’s shapes, before and after Harris corner detection. (a) Low SC, low corner count shapes; (b) high SC, high corner count shapes.

• The researchers looked at real-world recordings of people’s body motions or voices (in German) to determine if the spectral centroid of the voices and movements could be used to predict participants’ emotional arousal judgments. The researchers discovered that higher spectral centroids indicated higher emotional arousal judgments.

See Also

“Our results show how the spectral centroid, or the balance of high-frequency versus low-frequency energy present in sounds, shapes, and movements, allows us to express and understand emotional arousal,” adds Sievers.

A multi-sensory code for emotional arousal, Beau Sievers, Caitlyn Lee, William Haslett and Thalia Wheatley

Published: July 2019

Scroll To Top