What is in a sound?

Today's post is about something that we often take for granted, something which we pay no real attention to because the effects are an automatic assumption.

Of course, we are talking about sound.

Sound comes in many types, bitrates, formats, and channels. We can say possibly that the codecs for sound formats such as MP3 or OGG are the shell in which the essence are wrapped, and that the interpreter itself is what utilizes this in order to produce the audio. But what happens when you take it another step forward, into the third dimension?

In real life, our ears are placed roughly 6 inches apart and separated by our head in between. When acoustics in the air reach each ear (we'll say a stereo channel, left or right) the sound itself is shaped by the very structure of our head in ever so subtle ways. This in turn, the left and right channels of our ears, allows our brain to process a spacial location for sounds that we hear.

There is a form of recorded audio known as Binaural Recording which uses this characteristic of human hearing to it's advantage, and places small microphones inside a dummy head modeled after a human head. Some even take this further and model in detail the fake human ears in which the tiny microphones reside.

When recorded in this manner, it is possible to convey spatial position to the listener as though the sounds were really happening around them - but the effect only works for headphones. This in itself is because headphones are placed over the ear and isolate each channel to the listener in the same manner that the tiny microphones for the dummy head were isolated (earbud headphones do a poor job keeping this effect)

Obviously this effect of Binaural Audio is lost when listening on a set of standard speakers, as the acoustics are out of phase, but it does bring up a very interesting idea concerning virtual audio implementations.

When listening to a Binaural recording with headphones, your brain fills in the blanks and then informs you of the positions for each sound as they are located around you. So if there is a guitar playing in front of you, the acoustics in binaural audio would trick your brain into telling you there is a guitar playing in front of you. The same can be said for all space around you - left, right, front, back, up and down.

The down side to binaural audio (other than the headphones) is that since it is recorded via a stationary microphone setup, moving your head around does not change the locations appropriately of the sound locations. Instead it would seem that you are moving the entire room when you move your head.

Applying this idea, though, to a virtual world, we suddenly see the amazing benefits to having positional audio. Where in many cases today, virtual environments only offer four degrees of spatial awareness in audio (left, right, near, far) applying full spatial audio techniques to a virtual world would offer us the ability to hear with higher resolution.

As an example, let us say we have a chirping bird as a looping sound effect.

In a system with four degrees of spatial awareness (left, right, near, far) we would only be able to place said bird to our left or right and with a relative distance. With a full spatial audio algorithm, we would be able to hear with higher resolution, in that not only would we mentally place said bird to our right or left, near or far, but also above, below, front or behind.

Increasing spatial audio resolution to 8 degrees of freedom doubles the accuracy by which the brain can tell the listener where in a 3D space a sound is located. In this latter case, we are talking about the difference between knowing the bird is on our left or right and near or far, to knowing that same bird is on our right, behind us, about 25 feet up, and about 50 feet distance.

When utilized with multiple sounds, we in turn create a fully spatial soundscape for the user, and thusly a fully immersive environment. So then we take this one step further, because in a virtual environment we can move around.

With a spatial audio algorithm, the audio samples and locations would be dynamically adjusted as you moved around, so moving in the virtual world would alter this spatial soundscape in real time.

The effect would only work either with headphones or a surround sound setup, but when you listen only with a normal set of speakers the audio would sound like normal audio without the spatial effect. At best you may be returned to left, right, near and far (4 degrees) versus a full 8 degrees of audio.

If you have a pair of headphones handy, and are ready for a really amazing experience, we have provided a link to a list of streaming samples. One of our favorites happens to be the Virtual Haircut.

Binaural (Holophonic) Audio Examples - Listen With Headphones!

Until Next Time -

Andromeda Media Universe
Development Team