Researchers in the Department of Psychological and Brain Sciences have found that sounds, shapes, speech, and body movements convey emotional arousal the same way across the senses, suggesting that nearly anything can have an emotional tone, including art, architecture, and music.

The study, published this month in the Proceedings of the Royal Society B, set out to better understand whether there is a low-level mechanism that allows the human brain to decode emotional information from movements and sounds, the researchers say. They found that something called the “spectral centroid,” or the balance of high-frequency versus low-frequency energy present in sounds, shapes, and movements allows humans to express and understand emotional arousal.

“In a series of studies, we demonstrate that people automatically perceive the frequency spectrum of whatever is coming into their ears and eyes and compute the average—the spectral centroid,” says Thalia Wheatley, associate professor of psychological and brain sciences and principal investigator of the Dartmouth Social Systems Laboratory, where the research was conducted.

The spectral centroid is essentially a multi-sensory measure of spikiness, Wheatley says. “This is how people quickly identify the amount of emotional arousal in a person’s voice and movements, but also in abstract shapes and sounds, such as why spiky shapes seem to convey higher arousal than rounded shapes.”

The results help explain why Zen gardens and brutalist architecture, for instance, have very different emotional effects, as well as why things like clouds and lullabies seem to go together even though one is seen and the other is heard: We match them based on the spectral centroid.

While multi-sensory associations with emotions have been known for a long time, why they occur has been a mystery until now, says lead author on the study, Beau Sievers, Guarini ’10, ’18, a postdoctoral student of psychology at Harvard University who, at the time of the study, was a graduate student working with Wheatley in the Dartmouth Social Systems Laboratory.

Caitlyn Lee ’19, an undergraduate student in the Department of Psychological and Brain Sciences, and post-doc William Haslett, Guarini ’05, ’13, in the Department of Biomedical Data Science at the Geisel School of Medicine also served as co-authors of the study.

Amy Olson can be reached at amy.d.olson@dartmouth.edu.