To terribly, terribly misquote Nietzsche, those who were seen playing virtual reality space exploration game Irrational Exuberance were thought to be insane by those who could not hear the music.
Well, they certainly looked weird anyway. In videos on YouTube, players are nearly knocked off balance by the whoosh of comets. They lurch backwards as clashing boulders rain down solar wind. And though the visual stimuli is bright and flashy, the sound design is key.
“It has a violent, raw feeling to it …Your ears are literally catching the wind and adding all these bass frequencies that are rumbling around in your brain … You can’t ignore it,” said Brad Fotsch, who designed Irrational Exuberance’s visceral soundscape, which splits the difference between a Philip Glass composition and a demolition crew knocking your house down.
The emphasis on the sonic frequencies is unexpected. Typically, VR is thought of as a medium for our eyes and not our ears. However, Fotsch and his teammates at Buffalo Vision, who are scattered about LA and Chicago, believe there are vast universes of alien and unexplored sound just waiting to be uncovered with VR.
“The idea of spatially arranged music is something that hasn’t been done a ton,” said Fotsch. He foresees an opportunity for VR to upend the way people perceive and interact with music and sound. “It’s potentially a whole new aspect of entertainment, a spatialized music experience, which is different from traditional stereo-listening.”
Of course, you wouldn’t know it from merely looking at the game. Irrational Exuberance’s Prologue, which was released for the HTC Vive last month (the game proper is due out by end of year), is one of the stranger additions to the space exploration genre. Like any good work of surrealism, the piece is easy to describe but hard to make sense of. It all begins in a dark cave on some small planet in outer space, where the player is free to manipulate energy and peel back the brittle mysteries of the universe.
However, the system the game uses to communicate spatially-precise sound within a millimeter of accuracy may be its most profound secret. “If you could take a Matrix view of the game, you’d see all these little sound sources popping up and existing in the environment to create a background audio-scape as you move around,” said Ben Vance, the lead designer on the game.
Because the sound engine tracks the precise position of the player in the VR room, the sound effects grow louder or softer, interact with a variety of filters, and even rearrange themselves depending on the player’s movement. Although games like Grand Theft Auto wrote the book on environmental sound, “you’ve never had this level of precision tied to your body,” Vance said.
“But what does that thing sound like?”
The newfound musicality of the VR body can be identified in any number of cool, sci-fi moments in the game. For instance, the player’s hands have been given great noise-making capabilities. As the player brings her hands closer together, the left will interact with the right and hum in unison. And if the player wants to use her hands to do something else, say, petting the little blue crystals found scattered among the space debris, her hands will ring out in a different pitch. But no matter what, the sound of her hands will be modified by the tilt of her head.
360 degree spatiality also comes into play depending on whether the player is standing north, south, east, or west of the objects in the game. Take the portal that shows up halfway through the demo and warps the player to a different world. “There’s a big, powerful ‘Splgt-Splgt’ sound that vaporizes your environment,” said Vance. The trouble wasn’t so much coming up with a satisfying sound effect as the portal moves in location to the player’s head, but making it sound correct as the player moves through the center of it.
One of the biggest allures of VR is how the tech allows people to visit places that are impossible in real-life, but thinking up natural and convincing sounds that don’t spoil the moment can be a struggle, especially for strange worlds like Irrational Exuberance’s with few reference points.
“There are things you might find buried in some mysterious asteroid that come to life when you get close to them. But what does that thing sound like?” Vance said, emphasizing the tenuous link between what the mind can create and what the body can inhabit.
And while VR holds huge potentiality for reorienting our relationships to sounds, there are plenty of traditional sounds that no longer work, such as conventional movie scores and game soundtracks.
“The minute you start playing music in a VR environment, you break the reality, because a soundtrack isn’t really part of reality,” said Joel Corelitz, the game’s other sound designer, who previously composed music on the games The Unfinished Swan (2012) and Hohokum (2014).
“And because this medium is called virtual Reality it’s going to be a challenge to figure out how we define reality. Where is the line? The coolest thing is no one knows,” he said.
