Sound System Allows Deaf People to Experience Music Like Never Before

Sean Leonard is buckling me into, essentially, a sound system.

First, he helps me into a rubbery, black backpack that looks, as he points out, a lot like “Batman armor.” The pack lies flat across my back. Its straps go over my shoulders and across my ribs and belly, then tighten like a life vest.

Leonard plugs the pack into a tablet computer, on which he’s cued up a list of songs. Then he hands me headphones, which I'm only going to be wearing half the time.

My job today isn’t to hear the music.

It’s to feel it.

This is the SubPac, a wearable device that makes music tactile; the company calls it “physical audio.” While SubPac started as a gadget mostly for sound designers, it’s attracting attention from people who are deaf or have limited hearing, who are exploring how it can be used for music, dance and education.

Feeling sound, instead of hearing it, is a form of sensory substitution—using information from one sense as a proxy for another. (The same principle is at work when people who are blind use their sense of touch to read Braille.)

“We’re transferring energy directly to the body,” says Leonard, SubPac's executive vice president for strategic development. “And then your body is just doing the rest of the work.”

For someone who can hear, enjoying recorded music usually means listening through a speaker. A speaker works by moving an amplifying cone, which pushes the air around it, propagating pressure waves that eventually reach the ear.

But with a SubPac, users wear two tactile transducers right on their back, so they feel the music’s vibration both directly and through bone conduction, as the energy rattles their rib cages.

“It’s all about contact with the body,” says Leonard, “taking air out of the equation.”

OK, now I'm ready to rattle. Leonard cues up the Beatles’ “Come Together,” and I burst into laughter. The song’s iconic shoooooop-boom-boom-boom bassline radiates across my back like a supercharged set of Magic Fingers.

“It’s a beautiful bass line, so it feels pretty good,” Leonard agrees. “But do this for me as well. Take the headphones off. Plug your ears.”

Have you ever pulled up at a stoplight next to a car with the bass cranked and the windows rolled up, and been able to recognize the song from the vibration alone? It’s like that.

“I’ll give you a fun one,” says Leonard, cuing up A$AP Rocky’s hip hop cover of “Big Spender.” The song begins with a fake-out, a tinny recording of a female vocalist over a traditional jazz band, and the SubPac doesn't do very much in response. But when the bass finally drops, does it ever drop. I can actually feel it slither down my spine, as the percussion begins hammering me from my shoulders down to my kidneys and through my ribs.

“You’re feeling the lower lows down in sort of your solar plexus region, and you are feeling the higher frequencies up higher,” says Leonard. “It’s so precise you are actually able to feel the notes moving up and down your body.”

'An Earthquake Took Over My Body'

Shaheem Sanchez is a Los Angeles freestyle R&B and hip hop dancer who lost his hearing at age 4 but started dancing at age 11—he says he practiced feeling music by bopping his head along to the subwoofers in his mom’s car. He now regularly wears a SubPac to perform.

From KQED Arts: Empowering the deaf community through dance

"When I first tried it on, it felt like an earthquake took over my body,” Sanchez says in an email. “And that's a good thing!"

Without the device, he writes, choreography is harder—he has to rely on memorizing the beat and repeating the counts over and over.

"With the SubPac," he says, "it's amazing how you can feel it just like everyone else. You can dance to the same song for both worlds [the hearing and the deaf]. Deaf people will never feel left out.”

Jo-Ann Dean, a Los Angeles sign language interpreter and producer whose company, SIGNmation, promotes deaf talent and advocates for accessibility in the entertainment industry, helped outfit audience members with the packs at her monthly revue, the ASL Cabaret.

“I liked the idea that somebody onstage would also be able to relate to somebody that’s in the audience wearing the unit,” says Dean.

Their first SubPac-enabled show included a dance performance by Sanchez, a percussion act, and cast members from the rock musical Spring Awakening, which includes both deaf and hearing actors, singing Broadway show tunes.

This summer, Dean also organized a poolside disco during the DeafNation World Expo in Las Vegas. Deaf DJ Nico DiMarco—the twin brother of dancer Nyle—spun tunes, which were wirelessly transmitted to dancers wearing the vests.

“Instead of somebody just moving up and down, bobbing to the music like you would with the bass, or tapping their foot with the boom, boom, boom," Dean says, "now the shoulders are moving. The arms are moving. They are swaying with whatever the music is.”

SubPac has donated sets to The Muse Seek Project in the Dominican Republic, which has organized silent dance sessions and excursions for kids to listen to whale songs. The company has also partnered with Brendan Angelides, AKA electronic musician Eskmo, for his FeelHarmonic education program, which offers a “visual concert” for deaf and hard-of-hearing children.

From Sound Design to Deaf Culture

SubPac began its life as an aid for musicians and producers who wanted to feel bass they couldn’t easily hear. The first design, funded by a 2013 Kickstarter campaign, fits into the back of a chair, so a person working with audio software can simply lean back into it.

“Producing low frequencies is fraught with all kinds of complications and problems. It’s expensive, it’s loud, it’s hard to reproduce accurately across any environment,” says Leonard, who in a previous career life was a studio sound engineer for bands like U2, The Breeders and Frank Black. “So instead of having a big subwoofer, pushing lots of air, making lots of noise, being potentially affected by the environment, this gives you very accurate, high fidelity, reproducible reference points.”

But deaf artists quickly joined the SubPac user base. Deaf DJ Robbie Wilde was an early adopter, and this spring, SubPac outfitted the guest section for deaf Dancing with the Stars winner Nyle DiMarco with seated versions of the device, so the audience could feel the beat as he danced.

Today, SubPac has offices in Los Angeles, Toronto and at Playground Studios in San Francisco, a co-development workspace that features an indoor swing set and two-story curly slide. One of their goals is to find new ways simply to help people feel good, says Meredith Badali, who leads the deaf and hard-of-hearing initiatives for the company’s health and wellness team.

“The SubPac is a medium for art,” she says. “It allows you to express yourself and it allows you to share the gift of music and feeling with others. It adds another dimension to unlocking the world around you or breaking down barriers for communication.”

Amping the Reality in Virtual Reality

Badali likes to think of the SubPac as a vessel into which you can insert any cultural wedge—music, dance, video games, virtual reality experiences, movies. During my visit to Playground, I watched the most recent "Star Wars" and "Mission Impossible" trailers while wearing the vest. This time, it felt like having a chaotic fireworks finale erupt across my back, each rumble echoing an onscreen explosion, spaceship maneuver or feat of derring-do by a spy.

Then, Leonard offered me a go at a virtual reality game called Audioshield. Still wearing the SubPac vest and headphones, we added a VR headset and a pair of hand controllers, and dropped me into the middle of a (virtual) neon-lit arena. My task was to jump around deflecting oncoming blobs of light, catching the blue ones with my left hand and the orange ones with the right—all to the beat of Missy Elliott’s enormously bouncy “WTF (Where They From),” which engulfed me in the beat from both the headphones and the vest. VR labs often rely on subwoofers built into the floor or mounted under chairs to relay tactile sensations and create the illusion of “presence,” or actually being within the virtual environment. Moving floors work, but actually being shaken by the chest makes the virtual feel incredibly close.

Darin McFadyen, SubPac’s head of creative, who also doubles as the DJ and electronic music producer FreQ Nasty, envisions SubPacs adding a tactile element to augmented reality. Imagine, he says, playing a game like Pokémon GO or taking an interactive tour of a museum without having to keep an eye on your phone. Instead, the pack buzzing to the left or right could cue you which way to walk.

“We don’t want to wander around holding up phones to our faces,” McFadyen says. “That’s not something we are used to doing, it’s not something that a lot of people like, and we're already seeing people walking off cliffs and walking into cars and ridiculous things like that.”

But people are already used to wearing equipment on their torsos, everything from backpacks to baby carriers. And, as Leonard points out, the back offers plenty of real estate for sensory input.

Indeed, by the time I am done at SubPac, I have moved and vibrated and jumped so much that I have thoroughly sweated on all of their gear. But to Leonard, this physicality is a good thing, an indicator that SubPac is onto something that will help musicians use sound and feeling to create truly inclusive entertainment for people who can hear and those who can’t.

“We have an opportunity here for artists to really connect with their fan base,” he says, “to create things that literally move people, physically move people, reach out and touch people.”