Off The Record is a VR documentary charting the rise of Bhangra and the Asian Underground scene. I worked as Audio Director with No Ghost, a specialist VR studio in London. This was their first time working with archive material.
Credits
- Audio Director
- Sound Designer
- Technical Audio
The Development Process
Audio-driven immersion was picked to be the lead design concept for the piece, which meant I was deeply embedded within the development process. The team shared conceptual imagery of the three-screen VR cinema they had in mind. It was envisaged that the three screens would play out one seamless narrative.
My job was to design an immersive sound track that would draw the eyes and ears across three complementary montage films. I also had to implement this in Unity.
Technical Considerations
The question of how we would programme a triple projector system was a puzzle to be solved prior to designing the sound. A major constraint was that the three video players only supported spatialised mono and we needed to create an immersive mix. This necessitated synchronising ambisonic surround sound files with the mono synch.
I needed to structure and export a multi-speaker soundtrack glued together by time-coded events.
I collaborated closely with the programmer Miia Remhal from the outset, and between us we worked out how to do this.
In the end, I mixed out hundreds of small files with screen and timecode IDs embedded in the naming convention. This enabled Miia to hook the files into her projector system in real time. I used ambisonics on game objects placed around the listener to flesh out the surrounds.
For the nerds amongst you – see below for examples of how files were exported from Reaper.
Watch The Film (not in VR though!)
What you see here is a binaural render of the film and it is best appreciated in headphones.
Conclusion
Around fifty percent of the work involved in the design was structuring the stems for real-time playback over three screens. Another thirty per cent was cleaning up the archive sound, which was a bit of a mess. The only sounds that survived from the original material is the dialogue and voice over. Every single other sound you hear in the film above was recreated from sound libraries and foley recordings.
This was great project to be part of. No Ghost were a fantastic studio to work with, and it was a rare joy to encounter such good development practices. For best results, include your audio director from the start kids!