Thursday, 25 June, 2020 UTC
Summary
Strip away graphics and gestures, and audio is one of the most important parts of any experience. Good graphics go far, and mechanics make a world feel alive – but good audio brings is absolutely necessary. No audio is like a house without walls. Spatial audio is not as sexy as good graphics, but its impact far exceeds pixel counts.
This is an article about the improtance of spatial audio, and how recent innovations may be important in the future.
What is spatial audio?
Spatial audio is a step up from regular audio. Typically audio has very simplistical surround-sound qualities, such as left and right. The biggest streteches they can do is whether an istrument is playing to the left or right of an individual. Spatial audio takes the next step and lets musicians and developers simulate sounds above and behind people as well. An example may be playing a gunshot behind someone, or a plane flying above someone.
We’ve already seen some incredible work in the area. Oliver Kadel runs a great podcast on audio and immersive projects, interviewing key figures bringing their work to life (Oliver’s experiences running the podcast are worth a read). Perhaps my favorite is by Playlines and Harry Shotta, called Consequences; an immersive AR rap experience that’s part-concept album, part-silent disco, and part-close encounter with the British grime scene. Yes, you read that correctly.
It’s a real mix, but Muki Kulhan did a fantastic job. But what really helped it turn up a notch was the Bose AR glasses, which tracked when people are near certain location to continue the story. Location-based storytelling, enabled with spatial audio. Brilliant.
Having glasses with spatial audio means that people can hear voices around them, almost like a ghost, and checking out audio queues based on what’s said. The potential for immersive tech is immense.
So when Bose closed its AR platform, I was sad. The potential was there and used, but the lack of commersial use wrung the product up to dry. Then a week later, Apple announced that their AirPods Pro will update with spatial audio capabilities. Audio projects like Consequences may then move to Apple as they pursue their (very impressive) tech.
Last thought. Apple’s upcoming AR glasses aims to be as lightweight as possible. It’s not a stretch to think that, to save size and space, audio capabilities would be partitioned to AirPods Pro, and that the upgrade to spatial audio helps with their upcoming project line.
Apple showing spatial audio with the AirPods. Photo credit: Apple.
How does VR deliver sound?
It depends on the VR headset. The Oculus Quest, for example, has internal headphones which deliver contextual sounds during gameplay. The audio is very good, bringing people into games like Tetris Effect or Robo Recall. Others, like the Valve Index of HTC VIVE Pro, also offer similar audio qualities.
The best sound comes from the right headset. By plugging it in or connecting via bluetooth, most users can have an above-average experience by blocking out the real world. That said, there is the danger that people might be calling for you and you won’t be able to hear.
How important is high-end audio for virtual reality?
Immensely improtant. Virtual reality helps to bring people into new worlds, immersing people with swords, shields, or guns as they walk through a new location. High-end audio is the cherry on top of the cake, tying the great graphics and gameplay together like glue.
By comparison, poor audio cripples a good virtual reality game. Imagine wondering around the world and you hear rusty birds cawking in the distance, or an audio file that plays slightly too late when a sword swipes in front of you. Audio is as much part of the virtual world as the plans and the animals, and must never be sacrificed for the sake of an experience.
How will spatial audio impact multiple industries?
While at its early stages at the time of writing (2020), spatial audio is set to improve multiple experiances across immersive technology. Here are a few examples to cut your teeth into:
Entertainment
In cinemas, films are equipped with sophisticated sound systems that emulate sounds above or behind viewers. A great example of this is Dunkirk, where planes can rattle above and beyond viewers while they watch the war film. The same viewing experience can be brought home via earphones, which lets viewers get an even better viewing from the comfort of their couches.
That said, mobile devices are not the best way of viewing films. Nothing beats sitting back in a comfortable chair among friends, and watching the latest blockbuster on a massive screen among friends and family. But for those who are at home, the innovation is enough to spark some amazing work.
Raindance Immersive location for 2019. Photo credit: Tom Ffiske.
VR films
The same principles apply to VR films as well. Imagine looking at an immersive film from Raindance Immersive, with conversations happening all around a person. Having voices above and behind individuals contributes the the immersion of artistic pieces, hopefully driving some innovative items in the future.
Imagine solving a murder mystery in VR, roaming around a room in an Oculus Quest headset searching for clues. Then you hear a pitter-patter behind you, and you look bejhind to see mice crawling across the room. Or perhaps while playing a VR shooter, and you hear gunshots above you; a sniper taking aim at you. VR improves immersion, but spatial audio finishes the package.
Subscribe to Virtual Perceptions
Keep up to date with the trends and topics of the immersive reality industry, from gaming to healthcare and beyond.
Subscribe
Tom Ffiske
Editor, Virtual Perceptions
Tom Ffiske specialises in writing about VR, AR, and MR across the immersive reality industry. Tom is based in London.
The post Spatial audio and VR/AR: What’s the significance? appeared first on Virtual Perceptions.