One of the most talked about technologies today is virtual reality (VR). Though there are dedicated high end hardware and devices such as the HTC Vive and the Oculus Rift, the advance of smartphones has made these technologies affordable and accessible to mainstream consumers, as such, content distribution now reaches a wider audience compared to the early days of its adoption.
The emphasis on sound design, or good audio in general, has always been given the back seat in many productions including games, television, and film. Content producers know that sound is important but somewhere along the development process, the role of sound is reduced to bits and pieces that are thrown together as opposed to being appropriately crafted to work as a complement to the product. The most frequent reason for this is budgeting constraints. For games, the realization for good sound effects has readily improved, but still, due to budget constraints, most game development companies opt for using stock sound effects to breathe life into their games instead of creating their own unique sound. Although there are many good quality stock sound effects offered by various studios, sound designed specifically for a specific game will be more effective as well as memorable.
The growing popularity of VR and 360 videos has sparked new interest in sound applications. People are beginning to take notice and acknowledge the role of sound for this new alternate world they are exploring. Our brains need to compute and translate what we see and what we are hearing into something that makes sense. Having believable sound effects of splashes that tallies to the visual of a baby tyrannosaurus rex jumping into a pool of chocolate fudge is crucial in order for us to accept the authenticity of this alternate world. Carefully designed sound will keep us anchored to this alternate world.
When you are in the virtual world or alternate world, the sound is all around you as opposed to having it projected from two speakers in the front as when you’re watching TV. You don’t need to have special types of headphones to have immersive sound experience. Any normal consumer headphones are capable of delivering the experience. The difference lies within the technology of ambisonic audio. (Ambisonic audio is a 360-degree spherical or three-dimensional representation of sound around you.) The most cost-effective medium of delivery is via headphones and utilizing the concept of binaural audio. Binaural audio is based on the fact that with two ears we can accurately tell the direction of sound, the distance, and any changes in the properties of sound. Coupled with head-tracking data from the VR headset, listeners can hear changes in sound as they move their heads and experience their alternate reality without the feeling of disconnect.
What about surround sound? 5.1 speakers setup? Although surround sound technology allows sound to propagate around the listeners by means of channelling and directing specific sound to different speakers, it moves on a single line and does not represent height and depth. The listener has to always be at the centre in order to get the optimum effect of the surround sound. For audio engineers to mix for 5.1 surround, they need 5 channels of audio to be routed to each speakers. Ambisonic, at its simplest form, known as the first order of ambisonic, uses only four channels and it is speaker independent. It can be decoded to represent any number of speakers around us. So instead of routing and channeling sound to different speakers, in ambisonic mixing the focus is on the spatial placement of the sound source in the space. Ambisonics can be delivered with multiple speakers but the best and most practical use in conjunction with VR head gear is still a pair of headphones.
When I first experienced ambisonic audio, I felt that it was something magical! I just couldn’t believe that the same old headphones I used to listen to music and watching Netflix, could be given a new life and purpose without purchasing any additional equipment. This is what excites me the most. It is an experience that we can create and make accessible to everyone.
Virtual Barber Shop by QSoundLabs. Use headphones for accurate result.
There are two scenarios in which ambisonic audio can be implemented. The first is by using an ambisonic microphones while recording a 360 video. With this method, the 3D sound will be directly associated with the 360 video and can be published as is. The second scenario is recreating the ambisonic sound. Assuming that you would like to add or enhance the 3D sound of your video, or build everything from scratch for your VR games, you’ll then need to mix your various sound effect tracks to a four channel ambisonic B-format. The analogy would be something like mixing for stereo CD audio. You have your musical instruments, guitars, drums, vocals, but the end result is only 2 channels that can give a sense of horizontal spatial placement in a single audio file. The ambisonic audio mixing will give additional two channels that represent depth and height.
Ambisonic is not new. It was developed in the 1970s by Peter Fellget and Michael Gerzon. The applications, however, are more popular now due to the popularity of VR. For sound designers and engineers, it’s really an exciting time to explore, experiment, and apply production techniques to craft the best experience possible this new era of alternate reality productions. Today there is a variety software that allows mixing for ambisonic B format with ease ranging from paid to free. This enables sound designers and mixing engineers to quickly learn and adapt to ambisonic audio mixing. The challenge is not only just by recording and creating good quality sound alone, but to also dabble into psychoacoustic, to further drive the immersive experience and invoke emotional reactions. A virtual world is real if our mind is made to believe that what we see and hear is real.
Article Credits
Featured Image by : glebstock / 123RF Stock Photo
Content Images Copyright & References
Oculus Rift Wiki
https://en.wikipedia.org/wiki/Oculus_Rift
The Verge
https://www.theverge.com/2016/6/9/11893826/business-edition-htc-valve-vive-virtual-reality-headset
Sennheiser
https://en-us.sennheiser.com/microphone-3d-audio-ambeo-vr-mic
No Film School
https://nofilmschool.com/2018/04/rode-offers-its-first-ambisonic-surround-sound-microphone