Immersive Audio Camp.
The Experimental Immersive Audio Camp took place at Real World Studios in June 2023. The residency was aimed at mid-career professionals with some experience of immersive audio and spatial sound. We were musicians, sound artists, game sound editors and post production film mixers.
The camp involved a process of blind collaboration between participants who had not met before. Natalia Mamcarczyk and I were paired up by the project team at Real World and Watershed. Prior to meeting, all we knew about each other was that we had been through the same application and interview process.
In addition to the group of ten participants, the camp organisers had invited session musicians and experienced producers to work with us. We were blessed with access to many of Real World’s studio engineers.
The camp brief was to experiment and to mix a multichannel track over the weekend. Dolby Atmos was to be the final format.
First Steps
Prior to the lab beginning, Natalia and I agreed to work with a Soma Laboratories Lyra-8 synth. We also decided to incorporate vocals, to mix in the ambisonic format, use Reaper and write a song. When we arrived, we had these broad goals in place.
Inevitably, as the weekend went on, we had to drop some of those plans – mainly the one to use Reaper and Ambisonics – but these pre-camp creative decisions meant we arrived ready to start.
To kick off on the first day, we laid down a Lyra-8 track. We both had our hands on the dials as we recorded what instantly became the backbone of the track. As we both love Dub effects, we played the recording through a rather splendid Space Echo unit that Real World had knocking around. Cutting the saturated distortion and delay into the original recording gave us some textures and dynamics. Having decided on a theme for the lyrics, I bashed them into shape whilst Natalia worked out a mix template.
Widening the Collaborative Circle
Honing ideas at pace was hard work, not least because we were also grappling with complex new processes and fledgling software.
Real World musicians, engineers and producers were on hand to help. We were surrounded by talent and part of the learning process was figuring out who best to approach for particular advice or musical input. I learnt a lot about the art of producing that weekend. More about that here.
Compositional Engineering
Natalia and I wanted to work with recording arrays and to explore the translation processes between ambisonics and Dolby Atmos. Technical experimentation of this kind informed the aesthetic decisions we were making as we tried to build towards a final mix.
We wanted to use multi mic recording techniques at the camp. We recorded drums and the vocals spatially using this mic array (set up by Hans-Martin Buff and D&B Soundscape), capturing the spatial sound and reverberation of the beautiful wood room at Real World.
We also re-recorded the Lyra several times across multiple speakers and microphones to give us perspective and proximity effects. In effect, we built space into the recordings as we went – hence the idea of compositional engineering.
We had great fun flying the Lyra, drums and vocals around, recording multiple layers across multiple mics to be decoded in the mix.
Here are some examples of the same drum part recorded on different microphones within the array. You can hear the space baked into the recording.
This compositional engineering approach was designed to save us messing around too much with reverb and panning in Reaper. It also gave the weekend a feeling of physical creation rather than one of hunching over plug-ins, clicking with a mouse. The piece built in our minds as we collected and re-recorded elements.
Technical R and D
Half way through the penultimate day of the camp, the deadline for playing a mix loomed and our ambisonic version wasn’t communicating very well into the speaker system. Due to the way we went about recording (with arrays), we had numerous multichannel stems encoded with spatial information. and it was no surprise that decoding it all was very complex and problematic.
Natalia was at the camp fresh from her masters at Huddersfield where she specialised in Spatial Audio, particularly 3D microphone techniques, Dolby Atmos workflows, and High Order Ambisonics production. She had a very strong grasp of what she was dealing with but it was still a slippery beast.
Natalia says;
Familiar with both the IEM and Sparta toolkits, I was eager to explore the possibility of decoding Ambisonics (especially at High Order) onto a Dolby Atmos-compatible speaker array.
Using the Sparta Allrad decoder plugin, we could select the number of speakers and configure their positions on the azimuth and elevation planes. We adhered to the ITU positioning standards established for Real World Studios’ Red Room—now equipped as their own 7.1.4 Dolby Atmos Suite.
This setup didn’t come without challenges, as we faced numerous rounds of trial and error, adjusting parameters with guidance from Buff in the Dolby mixing room to get the sound just right. This troubleshooting process taught us a great deal and required us to think as both producers/composers and technical engineers.
As the only team at the camp working with Ambisonics, our project sparked discussions with other participants about whether different approaches could achieve similar goals.
Natalia broke new ground that weekend and she has documented the detail of how she used the mic arrays, mix set ups and speaker configurations. She did eventually create an Ambisonic mix … several weeks after the camp finished. This became the subject for her MSc research thesis, which focused on translating between High Order Ambisonics and Dolby Atmos in loudspeaker array and binaural delivery.
Back at the camp, we had to switch horses and complete a rough and ready Dolby Atmos mix,
Mixing for Dolby Atmos
Understandably, as a music studio, Real World were exploring Atmos. Dolby Atmos is a format that attempts to standardise playback of multichannel immersive audio across a wide range of devices. This can be understood simply as the pairing of calibrated mixing environments with calibrated speaker/playback systems (e.g headphones, Lap Tops, TVs). Dolby Atmos is fast gaining a footprint within the mobile streaming music ecosystem and has recently been incorporated into game engines. Apple has adopted Dolby Atmos for mixes on its Apple Music platform and an increasing number of tracks are mixed to Dolby Atmos/Apple Music engineering standards.
At the time of the camp – June 2023 – Dolby Atmos mixing tools were prototypical, with plugins still in active development. It was very much a make-it-up-as-you-go-along tool set.
Our planned route of Reaper and Ambisonics had put us into uncharted territory and beyond the existing expertise within the studio, focussed as it was on Pro Tools and Atmos.
We had set out wanting to use the open source world of ambisonics, Reaper and DIY. In reality, we were being asked to deliver into a (world class) studio that was wired very differently. Calibrated professional music studios and post production sound facilities are simply not constructed or connected the way studios are wired within academic and art environments. Trying to search out standards is part of audio engineering. As Natalia says; Through consultation with Hans Martin-Buff, who had collaborated with my lecturer at the time, Prof. Hyunkook Lee, we incorporated Prof. Lee’s resource combining speaker positions across various array standards. This reference helped us define our horizontal and height angular values precisely, creating our bespoke translation into a 7.1.4 system.
Immersive audio is necessarily technical. And the technical influences the creative process and the creative goals bend the boundaries of the tech. As with game audio development, tools and aesthetic choices can sometimes become indistinguishable.
Conclusion
I had gone into the camp wanting to be forced to be outside of my comfort zone, and I was. Breaking one’s own mould is always going to hurt a bit, but I came away with so much. We did things a new way, and this was what I had wanted.
For those of us working in production – films, VR and game audio, fast music deadlines – time is seldom a friend and it is hard to avoid formulaic processes when you produce to budget. Being productive and being creative are, in my experience, kind of co-dependent frenemies that live in constant turmoil together when you produce for a living . The better you get, the faster you work, the more you produce. Finding time to compose new and original music and search out collaborators can sometimes feel impossible.
Trying to find time to do something you haven’t done before is hard: a lot of headspace is used up when confronted with a challenge you are not sure you can yet meet. Any research process into the use of new mediums will create a kind of myopic feeling of navigating a long journey one finger nail hold at a time. Breaking new ground is hard when it is happening, but you make it easier for those that follow.
The camp was a gift for this. We used a vast amount of new technical approaches and I finally got to work with top music producers – something I have always wanted to experience. I still carry inspiration from this and take it forward as part of my creative approach. Natalia and I are still in touch and developing new ideas. We created a track we are both proud of: it was a tough but great experience.
Listen to the track here: