Forget 3D Screens—We Need 3D Audio, Like in Real Life

Some decades ago, a salesguy in a high-end audio shop badly misjudged my socioeconomic status and treated me to an ultrahigh-quality recording of an obscure jazz ensemble, played on a $10,000 audio system in an acoustically perfect room. I staggered out goose-bumped and hair-raised, a newly minted audiophile wannabe. I was sure that this was just the beginning of a journey into ever-more-amazing sound experiences. The equipment in that room consisted of glowing tubes in big metal cases, vibrating domes in massive wood cabinets, and spinning platters of plastic. No doubt technological innovation would one day shrink this clunky system into something small enough to carry around and cheap enough to avoid triggering the reckless-behavior clause in my prenup. More important, I was sure that even grander realms of audio quality lay ahead. By 2011, who could imagine what sort of incredible sonic delights would await? Technology certainly has come through in some ways. Today’s iPod Shuffle is so small that it is little more than audio-enabled jewelry. No complaints on the pricing, either; you can get a pretty good MP3 player for the cost of a newly released CD. There’s just one little snag: Today’s sound quality is miserable, worse than what I was listening to on my budget stereo 30 years ago. The biggest culprit in our sonic backsliding is the ubiquity of low-quality digital music files. “If you’re not going to listen to a high-quality recording, you don’t need a high-quality system,” says John Meyer, founder of the audiophile speaker company Newform Research in Ontario. Hey, tell my kids. They are all too happy to semipermanently install wads of plastic in their ears for the privilege of listening to near-terabytic playlists rendered in mediocre-at-best fidelity. The music and electronics industries have eagerly catered to our growing obsession with convenience, blithely sacrificing sound in the process. All the way back in the 1980s, audiophiles were pointing out that those newfangled digital CDs lacked the subtlety and warmth of the best vinyl recordings. And the most popular versions of today’s standard, the MP3 file, have just a fraction of the potential fidelity of a CD recording. The problem with MP3s is that they are “lossy,” which means they literally are missing some of the sound. When your brain hears sounds made up of multiple frequencies (as almost all music is), it tends to pay attention to whichever frequencies are the most readily perceived at any moment and largely ignores the rest. Most MP3 files simply leave out the subtler components of the music altogether—as much as 85 percent of what is actually recorded—in order to shrink the file size. In theory we should not much notice what’s missing, but in practice a careful listener will find the diminished quality hard to ignore, especially when playing MP3s on a high-fidelity home stereo. To my kids this blandness has just become the standard of what recorded music sounds like: They have learned to like their music uniformly loud and stripped-down to an in-your-face artificial clarity that does away with all the warm, rounded audio undercurrents. The good news is that the lab of Louis Thibault, director of Canada’s Communications Research Centre’s Advanced Audio Systems Group, is developing a superior way to encode music files. The technique involves plotting out how the music varies over time in frequency and amplitude, which results in a graph that depicts the music as a sort of rugged 3-D mountainscape. Visualizing a recording this way lets you describe the music in terms of geometric shapes instead of as a bunch of frequencies. That approach turns out to save a lot of file space, in the same way that describing a circle as a center point and a radius is more efficient than describing every little segment of the circle. “It looks as if we can reduce file size by about 50 percent compared with MP3s, with the same audio quality,” Thibault says. Turned around, this “object-based compression,” as it’s called, could provide much higher fidelity than that of a typical 16-bit MP3 in an equal-size file. Apple, meanwhile, is reportedly developing a new digital music player that can handle higher-resolution, 24-bit recordings, but who wants pricier, slower downloads that will make your existing music player obsolete? If Thibault’s compression scheme becomes standard, as he hopes it will, we could keep our 16-bit music players, and headphones could easily catch up; a decent pair of $50 earbuds already well exceed the potential of the music that gets poured into them. My kids may go into audio shock when they find out what they’ve been missing. But pumping up the fidelity of digital recordings only gets me back to where I started all those many years ago in the audiophile shop. What I really want is an improvement that will rock my sonic world, the way that sadistic salesperson did so long ago. So I got in touch with Karlheinz Brandenburg, who, in addition to being director of the Fraunhofer Institute for Digital Media Technology in Ilmenau, Germany, is also the audio technology legend who largely developed the MP3 file. Sure enough, Brandenburg has moved way, way beyond simply trying to get higher-fidelity recordings packed into smaller files. What he’s chasing now is spatially realistic sound. In real life, everything we hear is highly dependent on spatial orientation, for three reasons. First, your ear is shaped funny; second, the environment around you messes with all the sound waves that bounce around before they reach you; and third, your other ear is also shaped funny. Sound may reach your right ear first, then your left ear; a portion of the sound may be reflected by the wall behind you while another portion is partly absorbed by the coffee table in front of you. Every sound is uniquely filtered by that odd maze of flaps in your ears, and every little twist or nod of your head alters the whole aural picture. “Your brain takes all this information and extracts from it not just what the sounds are and where the sources are, but a sense of what the environment is around you,” says Agnieszka Roginska, associate director of New York University’s music technology program, who is also working on spatially realistic sound. You probably have never noticed how spatially challenged your recorded music sounds because you have never heard a recording that wasn’t. Conventional audio technology can capture and spit back every note, whisper, and rustle that was recorded, but it does nothing to recreate the complex directional variability of the original sounds. (Surround sound can provide only a crude sense of which direction a specific sound is coming from.) I would invite you, then, to visit Disney World’s Haunted Mansion, which features one of the most advanced 3-D sound systems in the world. The technology, based on the work of Brandenburg’s lab, surrounds a crowd of listeners with a ring of loudspeakers placed at intervals of a few feet. Each speaker puts out a slightly different version of the sound, acoustically re- creating the direction-dependent environment in which it was recorded. Or drop in on the Bregenz Festival in Austria, where another version of the system rings the 6,800-seat outdoor venue with 800-plus speakers, broadcasting live opera music in a way that puts everyone in the middle of the action. “You’re sitting outside, but it sounds exactly like you have great seats in a concert hall,” Brandenburg says. The audience even hears the sound follow the performers as they move back and forth across the stage. A home version could be available as early as next year for about $200,000. Not likely for me, unless I move in with Donald Trump. So the next big jump in sound quality is out there, but the price tag has soared far above $10,000—and even if I did have the money, I’d be stuck on the living-room couch to hear it. What I want to know is: When will I have it with a pocket player and headphones? Brandenburg has been unable to get the same effect with headphones that he can get with a roomful of loudspeakers. Roginska has better news. Her group has been developing mathematical models of how different ears filter different sounds coming from different directions. With that knowledge the team can build filtering schemes for headphones that make recordings sound as they would in the real world. When can I buy the technology? Roginska suggested I ask Veronique Larcher, an audio scientist at the Palo Alto, California, research lab of the headphone manufacturer Sennheiser. I did, but Larcher wouldn’t tell me anything about it on the record. I take that as a good sign. Companies clam up only when they think they might actually have a clue how to make something happen. Meanwhile, Roginska’s group is already putting its technique to work in a 3-D audio software program called AudioMaps, which relies on the gyroscopic sensors in some mobile devices to determine which way the device is pointing. For example, if you were listening to an audio tour of New York City, you could simply point your iWhatever at the Empire State Building and the narration would sound as if it were actually emanating from the building. If only I had an iPhone, and if only I ever went tourist attraction–hunting. And yet I feel strangely confident that any day now, the David H. Freedman Spatially Customized Headphones will be available for listening to the ultra-hi-fi, object-based-compression music files that will soon be cramming my pocket music player. Hey, it’s not as if technology has ever let me down before.

Forget 3D Screens—We Need 3D Audio, Like in Real Life

A backward march of audio quality has left  us listening to tinny, stripped-down MP3s. It’s time to show the kids what they are missing.

Newsletter

David H Freedman

David Pluckert

Published in:

First Look Inside a Black Hole

Do Smartphones Need to Get Smarter?

This Electric Powerboat Travels Above The Water

How Sonar Tech Is Solving Underwater Mysteries

Can Wearable Devices Save Your Relationship?

This Is Why the Inverted Filter on TikTok Makes Your Face Look Weird

With 5G, It's All About Speed

Shapeshifting Materials Could Transform Our World Inside Out

5G Has Arrived. What Is It and How Does It Work?

Meet Angus: A Robotic Field Hand for the Autonomous Farming Revolution

Meet YuMi: A Robot Nurse Built to Make the Rounds

This Skin Patch Could Add A Sense of Touch to Virtual Reality Devices

With a Floating Bead, This Device Makes Truly 3D Holographs

Creepy Music and Soviet Spycraft: The Amazing Life of Leon Theremin

Low on Juice: How Phone Batteries Shape the Rhythms of Our Daily Lives

These Robotic Shorts Make Everyday Strolls Feel Like a Moving Airport Walkway

Stay Curious

JoinOur List

SubscribeTo The Magazine

Forget 3D Screens—We Need 3D Audio, Like in Real Life

A backward march of audio quality has left us listening to tinny, stripped-down MP3s. It’s time to show the kids what they are missing.

Newsletter

David H Freedman

David Pluckert

Published in:

First Look Inside a Black Hole

Stay Curious

JoinOur List

SubscribeTo The Magazine

A backward march of audio quality has left  us listening to tinny, stripped-down MP3s. It’s time to show the kids what they are missing.