I Want HRTFs In My Games!

You’re the last man standing on your team and you’re going to have to work some magic to win this round. As you approach the objective, you hear footsteps from somewhere on your right. In a flash, you turn to face the sound but there’s nothing there. BANG! – you’re dead. It turns out those footsteps were coming from ‘above-right and slightly behind you’ but you couldn’t perceive that because this game’s audio sucks!

Whilst the sophistication of graphics in video games has steadily increased in the last decade, the same cannot be said for audio, which has remained stagnant and even regressed somewhat. Video games are an ideal case for integrating 3D audio technologies and yet they continue to be ignored.

As a long-time user of high-quality headphones, I find the audio in most modern 3D games to be quite boring and ‘flat’. The audio APIs that most studios are implementing within their games only provide a basic 2D panning algorithm when mixing for headphones. Such algorithms vary the intensity of sounds in each channel but that is only one aspect of sound localization. The others aspects, which are typically missing, include changes in frequency response and interaural time differences.

This is where Head-Related Transfer Functions (HRTF) come in. A game that implements HRTFs when mixing audio for headphones can provide an immersive, realistic, 3D sound stage in which the player can perceive the location of a sound source without any visual cues. Even sounds coming from somewhere above or below can be distinguished – all with just a regular pair of stereo headphones.

Solutions that are not the solution

There are several products available today that can apply HRTFs as a sort-of ‘post-process’. These products take an already-mixed 7.1 audio stream and apply HRTFs to it while mixing it down into a stereo stream for headphones. Examples of this sort of software include “SBX Pro Studio Surround”, “Dolby Headphone” and “Razer Surround”. These systems are limited in the following ways:

• Their 7.1 channel source material has no elevation information.
• The precision of the sound localization is limited to only 7 virtual sources of sound (provided by each speaker).
• They typically change the overall timbre of the sound too significantly to be acceptable.

For these reasons, this kind of HRTF implementation is of very little interest to me.

The real solution

To get satisfying results, HRTFs should be applied within the audio API itself – when all the original directional information is still available. For years, alternative OpenAL implementations have been the only APIs that provide HRTFs. “Rapture3D” and “OpenAL Soft” are examples of excellent implementations of the OpenAL API that provide great HRTFs.

OpenAL Soft is free software so, if you’re using the OpenAL API to handle audio in your game, there’s no reason not to use OpenAL Soft.

Rapture3D is commercial software and has been licensed by Codemasters Racing for use in several of their games. With a bit of tweaking, Rapture3D can also be used in a variety of other games that use OpenAL. The ability for end-users to swap the OpenAL implementation that a game uses is a big benefit of that API.

We are now also starting to see solutions emerging as plug-ins for the likes of Wwise, FMOD and Unity. “RealSpace3D” and “3Dception” are examples of these newer systems. Apart from the application of HRTFs to point sound sources, both of these products feature room modelling for sound reflections and these reflections are also passed through the HRTF filters. Incidentally, RealSpace3D was licensed by OculusVR.

Do better, dammit!

Game development studios should not content themselves with panning sounds across 5.1 and 7.1 speaker systems as a solution for 3D audio. Such speaker systems do not represent the ideal consumer-grade 3D audio solution. They actually suffer from several drawbacks:

• They can take up a lot of space and cause clutter from all the wiring.
• Depending on the room in which you’re setting them up, it can often be hard or impractical to place all the speakers at their ‘correct’ positions and elevations.
• You obviously cannot listen privately and the noise can annoy other family members.

Headphones don’t have any of these problems and you can enjoy amazingly clear and accurate sound if you purchase a high-quality pair. There is so much more potential for creating sophisticated, immersive 3D audio experiences for headphone-users and it’s about time that the games industry started spending more time on advancing the state of audio in games.

(Updated on March 7, 2015.)