This Outfit Can Help Deaf People Feel Music
The SoundShirt translates sound waves into vibrations on the skin, opening up the experience of music to the Deaf
In a London nightclub, under flashing lights, the twins Hermon and Heroda Berhane dance with a crowd of revelers. They are Deaf, and while they cannot hear the music, their clothing ripples and buzzes along with the sound.
“There are so many ways technology can include Deaf people and make our lives more inclusive and make us feel a part of society,” they tell me later in a cowritten email. They’re describing the experience of using a device called the SoundShirt, which was developed by the London-based fashion and technology company, CuteCircuit. “Music can lift you or a sad song can make you reflect, so to ‘feel the music’ was an emotional experience for us.”
Using actuators embedded into the fabric of the garment, the SoundShirt takes music and transforms it into a set of motorized vibrations. Different instruments are mapped across the torso and arms as haptic sensations, whirring and oscillating against the skin as a sort of tactile translation of sound. For a Deaf person, the SoundShirt’s designers claim, it is a way to truly “feel music.”
In CuteCircuit’s London office, amidst the skyscrapers of Canary Wharf, I try it on. There are no flashing lights and no crowded dance floor in this high-rise of buffed wood and floor-to-ceiling windows, but there is a filmed recording of an orchestra playing Felix Mendelssohn’s concert overture, “The Hebrides.” As the strings bloom and the percussion thunders, my gut vibrates beneath a set of cylinders woven into the fabric. It gets close to capturing the sensation of your stomach dropping at a rousing crescendo, albeit at the command of carefully curated mechanisms.
I am not Deaf, and can both watch and listen to the recording of the orchestra as the SoundShirt does its work. Having at least one other stimuli is crucial to understanding what is happening. “For Deaf users, it’s quite important to see the action in order to map what they’re feeling to what instrument is creating that sensation,” says Ryan Genz, one of CuteCircuit’s co-founders. In this way, the brain is able to draw connections between what it is seeing and the waves of motorized movement dousing the body.
The whole idea of “feeling” music is a deeply intimate prospect, and it’s interesting to learn that the roots of CuteCircuit’s creation stem back to an earlier device called the HugShirt. “The idea was to send hugs over long distances,” says Francesca Rosella, the studio’s other co-founder. “You put on the shirt, give yourself a squeeze, and sensors capture where you’re touching, how strongly you are touching, how long you are touching for. All this data could be broadcast to the other side of the world, to a friend also wearing a HugShirt.”
When developing the HugShirt in 2002, Rosella and Genz would run what she describes as “body storms,” inviting testers to embrace each other while they used red marker to outline the places hands tended to be put on other people’s bodies. “We had people literally hug each other for a very long time,” she explains. “Some of them eventually got married.”
Like the translation of words from one language to another, it’s never as simple as it might at first seem; bound up with the decisions of those that are doing the interpreting.
In 2016, the Junge Symphoniker Orchestra in Germany reached out to CuteCircuit, asking if the studio would be able to turn the HugShirt towards making its concerts more accessible to Deaf audience members. The studio agreed and set about mapping a symphony in touch, overlapping the hug patterns of its earlier creation with the instruments of an orchestra.
Hugging wasn’t the only consideration to locating sounds on the body. Genz explains that their next point of reference was the layout of the orchestra itself, with the high-pitched instruments closer to the front and lower frequencies, like the double bass and kettle drums, towards the back of the room. “So we mapped the shirt that way, that you have the high frequencies close to your face and moving to the back of the orchestra as it gets further away from your face,” says Genz.
Another consideration was how listeners tend to describe “feeling” music specifically in their gut. “If you’ve ever been to a club, you can typically feel the bass in your stomach,” says Genz. “If there are other instruments, like snare drums or beeps, you feel those more on your extremities. So we thought about the natural resonance areas of the body and tried to keep that mapping.”
Thanks to the work of 30 different actuators, the SoundShirt seems to writhe as the music plays, pulsing to the bass and tingling to the strings. The person wearing it certainly feels something, but do they feel the music?
Salomé Voegelin, an associate professor in sound arts at the London College of Communication, is skeptical. “When I listen to a Beethoven symphony in a concert hall, it does touch me. But that perceptive touch is very personal. It can get me in the guts, or take my breath. My body is contingent on the moment. Whereas the touching of this design is predetermined, in a way, by someone else.”
For Voegelin, mapping sound isn’t a straightforward task. Like the translation of words from one language to another, it’s never as simple as it might at first seem, bound up as it is with the decisions of those who are doing the interpreting.
“Within the translation of sound into impulses… on the shirt, onto your body, there is a designer who has an intent,” Voegelin tells me over the phone. “There is the technology itself, which has its own aesthetic and ideological remit. This is what a Beethoven symphony feels like, designed in this way, by that person, with this material, this fabric, and this technology.”
While she is cautious of CuteCircuit’s claims that the shirt can let people “feel” music, Voegelin sees radical potential in drawing lines between sound and touch. “Sound makes us think about touch in a different way. If we were to shake hands, we would say that my hand touched your hand. But in sound, it is far more complicated to say what was touched.” This fuzziness, she says, is a “sonic sensibility”; a way of thinking about the world that is different to the clear-cut visual sensibility we tend to use, which largely governs how we impose value judgements on objects and people in Western culture.
Perhaps what these devices strive for isn’t an equivalent sensation to sound, but a search for a totally different way of understanding the world around us.
“In this phone conversation, I can’t see you and I’ve never seen you before, but we are together,” she tells me. “A sonic sensibility brings us to a way of being with other people, with other things. It is a touch that prepares a very different social and political sensibility.”
Can technology live up to this radical vision? CuteCircuit isn’t the only company tapping into touch. U.S.-based startup NeoSensory, for example, says its Buzz wristband “detects the world around you and translates the sounds into rich patterns of vibrations,” with barking dogs and laughing babies feeling differently. It promises a way of being with other people that is more connected, tangible — and if you are Deaf — comprehensible. It could be of enormous use, but by aiming to “translate” sound in this way does it forgo the fuzzy “sonic sensibility” Voegelin identifies?
“One of the people we like to quote the most is Marshall McLuhan,” says CuteCircuit’s Rosella. “He said ‘perhaps touch is not just skin contact with things, but the very life of things in the mind.’” Perhaps “translation” isn’t the right word after all. Perhaps what these devices strive for, or at least should strive for, isn’t an equivalent sensation to sound, but a search for a totally different way of understanding the world around us.
For Hermon and Heroda Berhane, the way of being with other people, beneath the lights in a London nightclub, was determined by the overlap between sound and touch, music and vibration. They tell me it was an emotional experience, to “physically feel the music and be able to dance with everyone else in the crowd.” Whether it was the spinning motors or the dancing bodies, the sisters Berhane were touched by the music.
Update: A previous version of this article misstated where the Junge Symphoniker Orchestra is based. It is based in Germany.