A panel entitled “VR: The Last Medium?” kicked off the 2016 Versions Conference back in March with a discussion between Torfi Frans Olafsson (EVE Online), Jessica Brillhart (VR at Google), Neil McFarland (ustwo Games), and artist Rachel Rossin. The panelists, moderated by Kill Screen founder Jamin Warren, discussed the unique challenges presented by VR for those coming from other mediums, as well as thoughts on the birth of a new medium.
During the conversation, the group outlined challenges inherent with translating narrative from the flat surfaces of film into the experiential and exploratory realm of virtual reality. “Filmmaking is a medium that people are trying to push through, as if it’s going to be a 1:1 fit [with VR],” said Brillhart. While the comparison of VR to film feels apt—as both visual-based mediums allow a creator to present moving images with defined start and end points—upon further examination that relationship breaks down. As Brillhart listed the numerous problems, such as the proximity of performance and forging an audience relationship, the crux of the issue boils down to a single challenge: editing.
Film’s frame allows the director, cinematographer, and editor to control what the viewer does (and does not) see, but expecting that level of authorial control in a medium predicated on the user being wholly surrounded in an alternate world is, in Brillhart’s words, “wrong… it doesn’t make sense.” To Brillhart, “[in film] it’s frame to frame, and in VR it’s more like world-to-world,” referencing Walter Murch’s In the Blink of an Eye (1995) as a touchpoint. “The cut… represents a total and instantaneous displacement of one field of vision with another,” wrote Murch.
“we don’t usually blink and enter an entirely different environment.”
Beyond serving as functional transportation from place to place and advancing the plot—charmingly referred to by Murch as “cutting out the bad bits”—editing in film allows the filmmaker to guide the eyes; it lets visuals tell the story. It can affect big changes such as which shots are kept and which are left in, and more nuanced alterations such as the order in which shots appear. It’s a visual language, one developed over decades, and that works as audiences are naturally equipped and inclined to fill in the gaps themselves. It’s how we’re able to follow the split-second shot lengths of Janet Leigh being stabbed in a shower or Jason Bourne fighting off an assassin.
However, in virtual reality, the editor’s option to push and pull the audience’s attention with short and long cuts, or to rearrange shots to detail an event are largely taken away. When the viewer is experiencing the media in an all-encompassing visual space, a jump cut or quick change in camera angle would be, at best, horribly jarring, and at worst, cause the viewer to be physically ill. The question that editors working with virtual reality have to answer is how they can shape their narrative vision without complete control over the viewer’s eye.
Another filmmaker involved in the search for that answer is VR documentarian Danfung Dennis. He also alluded to Murch’s seminal work when discussing with me the fundamental editing choices in a recent work by his production company Condition One, titled In the Presence of Animals. “We were still fading to black in between the different shots and it’s it feels a little bit like a Band-Aid. It isn’t the final solution to editing [in VR], but it’s less jarring than a hard cut,” Dennis said. “Hard cuts are always pretty jarring—it just never really happens in our real life except when we blink. We don’t usually blink and enter an entirely different environment.’”
In much of the same ways that the average shot length of an early Hollywood film is significantly longer than the films of today, Dennis explained that his team has discovered that with today’s audiences, shots as long as 20 or 40 seconds are comfortable, even holding for a minute depending on the content. Condition One’s most recent release, Factory Farm, contains a three-and-a-half minute long shot.
Given the difficulty of locomotion in 360-degree video—due to hardware limitations inherent in stitching together moving footage from multiple cameras, as well as potential motion sickness from the audience—the viewer is confined to wherever the filmmaker’s camera was located on set. That means the director or editor still has some control over what the viewer sees in a VR film. However, technological advances in positional-based VR mean the viewer can now move around and actively explore a virtual environment. Authorial control in such a case would appear to be even further removed from the film maker.
Saschka Unseld of Oculus Story Studio feels it’s about establishing a balance between freedom and guidance. “The most important thing is to recognize in an experience when it is time for the viewer to explore by himself and when it is time, as a narrator, to finally tell them a specific story.”
Likening overt mechanics like audio cues in VR to obtuse film tropes such as overwrought narration or zoom-in shots, Unseld discussed how filmmakers are still feeling out how to translate their vision to virtual worlds. “It’s about finding that middle realm where you gently guide someone without forcing them yet don’t lose them,” he said.
Forced narration in a film can (poorly) serve to advance the plot yet ruin the audience’s engagement with the story; in the same way, it’s going to take time for audiences to be able to fully internalize this new visual language and innately understand how to experience VR content. There are growing pains for filmmakers and audiences alike.
“consider the body of the audience”
“It’s a mutual hand-in-hand [process],” said Unseld. When creating Story Studio’s first film Lost, his team understood they were creating a film that was likely to be the viewer’s introduction to VR and constructed it in a way most easily digestible for such a new audience. “It’s going to be a mutual growth of us trying things [in order] to tell a narrative in editing, and the audience [is] just learning about it.”
Emily Eifler of virtual reality research group EleVR brings a different perspective, likening VR editing to choreography. To her, VR content is a somatic medium, closer to interactive theater where the audience must participate rather than passively view. Creators must “now consider the body of the audience, not just the body of the performer” when making editing decisions, according to Eifler.
As with all burgeoning creative media, finding editing’s place in VR falls to artists, storytellers, and audiences as various approaches are trialled and reworked. From this, gradually a collective understanding of how to successfully convey narratives should emerge. This isn’t a problem inherent to virtual reality, but one that we are currently able to experience as early experiments find their way through trial and error until working theories and practices are properly established.
Immersion has been held as the high water mark for what VR should achieve in the mind of the viewer. To Efiler, this is far from a new concept, citing frescoes of Dionysian cults and the Battle of Sedan panorama as examples that every medium is capable of inspiring awe in our imaginations. Until VR finds its footing, to borrow Dennis’s phrase, Band-Aids will have to suffice in order to tell stories.
Unseld found the words to sum up virtual reality’s current state: “we’ve basically lost the editing room, and we’re trying to figure out very hard how to get that back.”
