I Want HRTFs In My Games!

You’re the last man standing on your team and you’re going to have to work some magic to win this round. As you approach the objective, you hear footsteps from somewhere on your right. In a flash, you turn to face the sound but there’s nothing there. BANG! – you’re dead. It turns out those footsteps were coming from ‘above-right and slightly behind you’ but you couldn’t perceive that because this game’s audio sucks!

Whilst the sophistication of graphics in video games has steadily increased in the last decade, the same cannot be said for audio, which has remained stagnant and even regressed somewhat. Video games are an ideal case for integrating 3D audio technologies and yet they continue to be ignored.

As a long-time user of high-quality headphones, I find the audio in most modern 3D games to be quite boring and ‘flat’. The audio APIs that most studios are implementing within their games only provide a basic 2D panning algorithm when mixing for headphones. Such algorithms vary the intensity of sounds in each channel but that is only one aspect of sound localization. The others aspects, which are typically missing, include changes in frequency response and interaural time differences.

This is where Head-Related Transfer Functions (HRTF) come in. A game that implements HRTFs when mixing audio for headphones can provide an immersive, realistic, 3D sound stage in which the player can perceive the location of a sound source without any visual cues. Even sounds coming from somewhere above or below can be distinguished – all with just a regular pair of stereo headphones.

Solutions that are not the solution

There are several products available today that can apply HRTFs as a sort-of ‘post-process’. These products take an already-mixed 7.1 audio stream and apply HRTFs to it while mixing it down into a stereo stream for headphones. Examples of this sort of software include “SBX Pro Studio Surround”, “Dolby Headphone” and “Razer Surround”. These systems are limited in the following ways:

  • Their 7.1 channel source material has no elevation information.
  • The precision of the sound localization is limited to only 7 virtual sources of sound (provided by each speaker).
  • They typically change the overall timbre of the sound too significantly to be acceptable.

For these reasons, this kind of HRTF implementation is of very little interest to me.

The real solution

To get satisfying results, HRTFs should be applied within the audio API itself – when all the original directional information is still available. For years, alternative OpenAL implementations have been the only APIs that provide HRTFs. “Rapture3D” and “OpenAL Soft” are examples of excellent implementations of the OpenAL API that provide great HRTFs.

OpenAL Soft is free software so, if you’re using the OpenAL API to handle audio in your game, there’s no reason not to use OpenAL Soft.

Rapture3D is commercial software and has been licensed by Codemasters Racing for use in several of their games. With a bit of tweaking, Rapture3D can also be used in a variety of other games that use OpenAL. The ability for end-users to swap the OpenAL implementation that a game uses is a big benefit of that API.

We are now also starting to see solutions emerging as plug-ins for the likes of Wwise, FMOD and Unity. “RealSpace3D” and “3Dception” are examples of these newer systems. Apart from the application of HRTFs to point sound sources, both of these products feature room modelling for sound reflections and these reflections are also passed through the HRTF filters. Incidentally, RealSpace3D was licensed by OculusVR.

Do better, dammit!

Game development studios should not content themselves with panning sounds across 5.1 and 7.1 speaker systems as a solution for 3D audio. Such speaker systems do not represent the ideal consumer-grade 3D audio solution. They actually suffer from several drawbacks:

  • They can take up a lot of space and cause clutter from all the wiring.
  • Depending on the room in which you’re setting them up, it can often be hard or impractical to place all the speakers at their ‘correct’ positions and elevations.
  • You obviously cannot listen privately and the noise can annoy other family members.

Headphones don’t have any of these problems and you can enjoy amazingly clear and accurate sound if you purchase a high-quality pair. There is so much more potential for creating sophisticated, immersive 3D audio experiences for headphone-users and it’s about time that the games industry started spending more time on advancing the state of audio in games.

(Updated on March 7, 2015.)

6 thoughts on “I Want HRTFs In My Games!”

  1. Great article!
    Can you recommend a set of high quality headphones?
    I want to have the best HRTF experience I can with the Rapture3D I just purchased.
    Also, some people say the HRTF experience is supposed to be better with IEMs (In ear monitors).
    Do you have any thoughts on that?

    1. I personally use my Sennheiser HD 800 and it sounds great. I’ve previously used an HD 518 and an HD 280 Pro and I remember them sounding great as well. I don’t think my experience is broad enough to be able to make a recommendation to you. It’s plausible that your choice of headphones could very marginally affect the HRTF experience that you get but Rapture3D comes with six different HRTF profiles so I’m sure you’ll be able to find one that sounds good for you.

      I imagine the theory with IEMs is that you’re avoiding introducing your own ear’s HRTFs over and above what’s in the sound coming from the speaker. I can’t comment on this from experience because I don’t have any IEMs but I must say that I doubt it has a huge influence. Even with my huge HD 800s, the sound is directed straight into my ear by the angled driver.

      I would prioritize comfort and sound quality when selecting a pair of headphones.

  2. I hear you man… I would trade EAX for proper HRTF audio any day of the week. Back in olden days, Thief the Dark Project’s sound rocked your socks off if you had an A3D capable sound card and good earcups.

    http://en.wikipedia.org/wiki/A3D

    Since then, Creative brought Aureal into bankruptcy, and now 15 years later we are almost universally stuck with console-ported positional audio of lower quality than the previous decade (fifteen years later!).

    I can understand why, to some extent; sound is difficult to sell, as it needs good earcups at the correct volume with distortion-resistant cabling, plugs and sound card sockets. It’s easier to notice graphics at 640×480 resolution instead of 1920×1080, but it’s more difficult for the lay man to distinguish good HRTF audio from basic 3D positional audio if their stereo headset is of poor quality or at a too low volume.

  3. Hi Matthew,

    Seven years, it took seven years to get close to what you was looking, are you happy now?

    Yes! the PS5 just launch this week and, oh boy, HRTF audio it’s gonna be a mayor star in the near future, I found more about you in your youtube channel and I really feel you are a truly visionary.

    I hope you are doing great.

Leave a Reply

Your email address will not be published. Required fields are marked *