This is cross-posted from the Immerse newsletter.
At Tribeca Film Festival 2018, the virtual reality expo was divided into three parts: an arcade of varied works in varied formats; group viewings of synched headsets preloaded with 360 video programs; and Storyscapes, five high-end virtual reality installations in a competition. Some were premieres, others versions of work previously on the circuit, and overall it was a great snapshot of a field that’s burgeoning with experimentation.
Together, they sparked a number of questions:
How are we so smart but so dumb? This is, apparently, a species smart enough to create multiple ways to create virtual reality with the latest technology, but not smart enough to notice that things like nuclear weaponry, climate change and racism are hazardous to our health and future as a species. So we’re using virtual reality to show us realities we should be noticing without it. OK.
Where’s the evidence? The argument for VR is that experience is memorable, creates empathy, makes people reflect and leads them to act. But we’re mostly not taking any measures to find out if it works any better than all of the other things we’ve been using to point out the obvious to ourselves. At Tribeca, only the occasional project was doing any testing. Courtney Cogburn, a social work professor, is working on measuring the effectiveness of 1000 Cut Journey, in which viewers can become a black man as a kindergartner, teen and adult, experiencing daily racism. Maybe Navid Khonsari’s Hero, an installation that combines VR with other sensory cues including dust, wind and heat to simulate a bomb attack in Syria, will do some testing of the proposition that it can be used to address PTSD. Participant Productions poured resources into This Is Climate Change, Danfung Dennis and Eric Strauss’ four-part mini-series of mini-360 films on climate-change crises, without planning any testing. But why experiment in a new medium that already requires individual access, without finding out what’s going on?
Where’s the ethics? Suppose VR is really powerful, persuasive, and empathy-conducive. Then responsibility for user experience ratchets up, doesn’t it? But ethical questions in VR are, if possible, even less well formed than in documentary filmmaking, where there is still no shared standard akin to what exists in journalism. I don’t have any answers, but no one really even seems to be asking questions. For instance, Hero and Michael Muller’s Into the Now are two VR projects that put viewers in virtual harm’s way. In the Now is about overcoming fear of sharks by swimming with them; it’s a prototype for a project to address phobia and PTSD. Should they warn people what level of intensity is ahead? Is it enough to explain what the installation is about? Should care be available for someone accidentally triggered? And if this is actually a prototype for a medical procedure, has anyone run it past an Institutional Review Board? Human subjects fall under special ethical guidelines, especially in medical trials.
Users do have questions. One of them asked Evette Vargas of Digital Design about the potential effects of adding advertiser pitches to interactive storytelling. (The men on the panel took a pass on this question.) Her answer was an echo of what happens whenever I ask about ethics. She said, “The creators take ethics seriously. It’s a tremendous responsibility, we think about how this affects the user experience emotionally and spiritually.” But this platitude assumes we already know what ethics questions to ask, and that we agree on the approaches to address them. Oh, and by the way, as this Tribeca panel showed, advertisers are eager to imbed their ads in your interactive viewer, or, if you will, viewzer. (Buzzword alert.) The company Wirewax is there to help, enthusiastically.
Where’s the improvement? We’re not waiting for technology to improve, before we move along to the next challenge. Which might be adding more sensory touches, to increase the intensity and the reality. Or increasing the peripheral vision on the headset. Or making interactivity even easier within the scene. But we’re not waiting to get an image that doesn’t look so blurry in lower resolution, or to create a downward-looking experience that isn’t nauseating. If we are, it’s not showing up at the user level yet. But hope springs eternal.
What’s the language? If you want to do storytelling in VR, you’re up against new questions about how to structure the experience. As Eric Strauss noted at Tribeca2018, the grammar and syntax are not stable yet. This is Climate Change uses straightforward narration over vivid emotionally complex experiences, where you can look where you like but are given hints about where to look. “You’re letting the viewer explore, you’re sharing the narrative,” Strauss said.
What is VR anyway? Is VR a storytelling medium? An experience? An art installation? Check out the dizzying collection of categories that Kamal Sinclair has devised to answer that question. What you think it is conditions what you do with it. Many creators at Tribeca2018 talked about conceiving the experience as a work of art in the installation mode. Lisa Jackson, the Canadian Native maker of the National Film Board project Biidaaban: First Light, has worked in a variety of forms. She embraced the notion of art installation to define Biidaaban, which brings viewers into a seemingly post-apocalyptic, abandoned downtown Toronto where Native dwellers experience the wholeness of the world with ancient texts. It’s elegant, simple, clean, and deeply moving. Remarkably, it’s also weirdly optimistic. “For many of you, the apocalypse is now,” Jackson said. “For Native people, the apocalypse has already happened. This is not post-apocalyptic. It is indigenous futurism.” Laurie Anderson and Hsin-Chien Huang’s Chalkroom is another enhanced installation-style experience. Users fly through a constantly shifting environment where walls and pillars covered with words and drawings hover in a vast universe. It’s delightful, whimsical, improvisational, and gives users an experience of opening up to opportunity. For some makers, VR is a kind of museum or virtual museum experience. Gabo Arora and Saschka Unseld’s The Day the World Changed puts users in the train station that was ground zero for the Hiroshima atomic bomb; users learn personal histories through found objects. A coda to the experience warns about the dangers of proliferating nuclear weapons. For some makers, it’s a different experience of performance. Terrence Malick directed Together (a collaboration with Facebook), featuring two miming dancers, giving users the opportunity to be onstage with them. (It made me nostalgic for Pina, Wim Wenders’ wonderful user of 3D to put viewers onstage in a performance of avant garde choreographer Pina Bauch’s work. And I only had to put on 3D glasses, not a lunchbox for the face.)
Is the novelty value enough? Anyone who has been to the VR/Immersive/Interactive section of any festival knows that it’s a crapshoot whether you’ll be able to see enough to draw any conclusions. You wait in line for the few headsets or slots at the installation, just in time for the system to crash and need a reboot. By the time you’re through, you lost your position at the next spot. And so on. How is this scalable to a viewer experience that can make a difference? Oculus’ VR for Good program teams up makers with nonprofits, and is a good place to look for answers. Their work is focused on 360 video, which can be easily done with headsets and doesn’t need a whole installation to accompany it. But you’re still bound to headsets and smartphone apps. Could these stories be told in a less cumbersome way? Everyone I’ve talked to from the nonprofits involved says, S ure we could, but—the novelty and promise alone make this is a powerful way to communicate, especially with donors, funders, and influencers who need a special something to bump it out of the ordinary. That can be important.
Where’s the memory? So much work at this point in VR is being done on the fly, tweaking programs by hand, and there are so few standards that any ability to recall even the recent past work is extremely difficult. Previous generations of VR go obsolete and when they do, go extinct. Digital art in general has a short shelf life, and museum curators have to be part-time software engineers to rejigger it for display. VR’s experimental era could be really hard for anyone later to experience again. Scholars, artists and users all suffer when you can’t access the past of a form.
As this form struggles into its tween years, all these grown-up questions get more and more real. The Tribeca2018 showcase demonstrated that many are willing to undertake the experimentation that make those questions relevant and worthwhile.