Thoughts on getting immersive media right.
Earlier this year, I had the pleasure of consulting on a VR project set in China's Mogao Caves - a UNESCO World Heritage Site of Buddhist grottoes in Dunhuang that were first dug out and decorated in the fourth century, as hubs of meditation, education and culture. The Cave, directed by Qi Zhao with art direction by Wen Feng, was recently nominated in the Immersive Non-fiction category of the International Documentary Film Festival Amsterdam's DocLab competition.
My advisory role focused on the immersive gestalt of The Cave: the interaction patterns (conceptual models) and techniques (implementations). As oft noted by VR creators, the project was a profound learning experience for all involved. Immersive media has a way of amplifying everything from the experience itself to the development and production challenges.
"Rule of Three" fan that I am, I distilled the learnings into three points that I consider keys to compelling VR...
1. Focus on the User
A firm grasp of the fundamentals of human perception is a must. It's not necessary to be a psychologist or a behavioral scientist, but it doesn't hurt to draw upon the insights of such experts either. Jason Jerald devotes a quarter of “The VR Book” to an overview of human perception, and Nobel laureate Richard Thaler's “Nudge” is an excellent book on influencing human behavior. Mastering "the art of indirection" in VR requires a keen understanding of perception and the skillful application of influence.
Minimizing sickness is a must. A poorly designed website can be aggravating, and a bad film may make you figuratively puke, but slipshod VR will literally make you sick. A bad initial VR experience can be impossible to overcome. Conversely, "VR sickness" is a phenomenon that some people adapt to over time (like getting your "sea legs"). Given that VR developers spend more time immersed than most, we should be mindful that novice users may be more sensitive to ill effects. In particular, there should be no conflict or disconnect between the visual and vestibular senses (between what you're seeing and what you're feeling).
Finally, you need to give people a reason to strap a clumsy piece of hardware on their face, and hopefully commit to the purchase of a VR kit. Irrespective of the medium, why would someone want to experience what you're creating? Why does your experience need to be in VR? What immersive and interactive enjoyment does your experience provide? Why will it leave the user wanting more? It's essential to address these questions in theory during development, and then validate your assumptions during production.
2. Keep it Simple
In VR, you must prioritize performance above all else. This means reducing latency by minimizing end-to-end delays in your system: everything from the hardware speed to the content load to the pipelining process. Keep in mind that inconsistent latency can be more troublesome than constant latency, since the former makes user adaptation difficult. While latency compensation techniques (including prediction) can be applied to reduce perceived latency, these techniques can backfire with large latencies. Given that end users may immerse themselves with various VR hardware, the aspect you have the most control over is your content load.
On that front, keep it simple. Focus on interaction over aesthetics. Make sure that your designers understand the unique affordances and challenges of the VR medium, especially if they come from film/video backgrounds (but even if they come from games). There's no point in creating beautiful character and environment designs that bring the system to its knees and send the user to the toilet. Don't default to clunky realism when there may be a more elegant and enjoyable symbolic approach. Consider how the responsive abstraction of Pong still beats most of today's more realistic games.
Even if the system can handle the asset load, you don't want to bombard the user's senses. Keep visual, audio and haptic information within a comfortable (and safe) range. VR experiences have an immersive intensity that can be compelling at best and overwhelming at worst. Visual cues should be clear and kept away from the eyes to avoid convergence issues. Darker scenes are generally more comfortable, and flashing lights should be avoided. While sound effects, music and sometimes narration can enhance the immersive experience, they can be annoying and even painful beyond a certain volume or when clumsily overlapped.
3. Experiment and Iterate
Ultimately, you must replace VR theory with VR practice in order to validate your assumptions. On the Mogao Caves VR project, there were surprises on every front - even as we tried to be mindful that we were working in a new medium and shouldn't take anything for granted. Experimentation, fast feedback and digestion of criticism are necessary for the success of your project and for the future of VR in general.
Validating assumptions means getting your team into a minimum viable product (MVP) as soon as possible, followed by adjustments and test sessions with fresh VR users. As the technological framework, creative content and interaction patterns are refined, expand your testing pool to include a broader demographic range within the appropriate context of the work. Don't assume that what is intuitive, comfortable and enjoyable for you as a VR creator will be so for everyone else. You're probably already immune to VR sickness and a fan of VR technology. Most people are not.
This third wave of virtual reality endeavor provides creators with the opportunity to explore and define a new vocabulary of immersive entertainment. The current state of VR is akin to the early days of cinema: a mature language of virtual reality has yet to evolve. We still speak of VR in terms of what came before - games, film, television - while wrestling what VR can and should be. Breakthroughs will come about through experimentation and iteration, not by dumping existing content into a new space. Developing VR in VR (rather than with the tools of other trades) is crucial.
Whether you're working in fiction or non-fiction, it's an exciting time to be a storyteller. We are laying the groundwork for future generations of "VR natives" who will grow up with immersive media and work out its true potential.
Kevin is the author of AWN's Reality Bites blog, his musings on the art, technology and business of immersive media (AR, VR, MR) and AI. You can find Kevin's website at www.kevingeiger.com and he can be reached at holler@kevingeiger.com.
