‘Walk in China’ VR Trilogy

As previously profiled on this blog, I’ve had the pleasure to serve as a consultant on Beijing-based Impact Eyes’ virtual reality trilogy, Walk in China – an immersive ode to Chinese cultural treasures. Walk in China officially premiered in the virtual reality section of the 16^th China International Film & TV Program Exhibition this September in Beijing to keen interest and rave reviews from expo attendees and visiting officials.

Following are some photos from the inaugural event, with context and commentary from Golden Horse-winning producer ZHAO Qi and Disney veteran director Wen FENG.

—

ZHAO Qi on the Walk in China trilogy…

China’s “Belt & Road” initiative is focused not only on economic cooperation, but also on cultural exchange. Just as the Silk Road carried ancient Chinese civilization to the outside world, so today the Belt & Road initiative conveys China’s cultural heritage to people far and wide. Advances in technology have spawned a proliferation of communication media. Through their convergence, these media offer a fresh and interactive immersive experience. Increasingly, audience participation is playing a crucial role in the dissemination of information. Against this background, we have produced the Walk in China VR trilogy.

Virtual reality immerses users in interactive experiences that transcend time and space. Our cultural project embraces a synthesis of laser scanning, photogrammetry, artistic innovation, plot construction and interactive design. The VR handset conjures a virtual space where users can examine the minutest details of the environment. The accompanying text and voice commentary – which can be activated at will – provides a deeper understanding of the relevant scenes and storylines. Users gain full access to all content generated in the virtual space, thanks to the subtly layered perspective of virtual reality technology.

ZHAO Qi on The Cave …

The Dunhuang region, once a vital link on the route from Asia to Europe, has throughout history been a pivotal trade hub between the East and West. It was the gateway through which Buddhism entered China’s heartland, through which the world’s major civilizations met and merged, through which art, technology and trade thrived. The images at Dunhuang’s Mogao Grottos are an illustrated historical chronicle that have drawn admiring visitors from across the globe, but are currently under preservation protections. When presenting the evocative and sensitive imagery of the Mogao Grottos, conventional media such as linear video have proven inadequate. So we decided to use more interactive and immersive methods to introduce Cave 285 – a renowned site that is rarely opened to visitors.

The laser scanning and photogrammetry equipment, coordinated by a sophisticated measurement system, gathered comprehensive three-dimensional data on Cave 285’s interior, down to the tiniest details of the surface texture. The data was then fed into a point cloud system for mapping and restoration. The resulting 3D geometry scans and 2D texture imagery were then combined to generate a precise, unmolested virtual replica of Dunhuang Mogao Cave 285.

When you don your VR headset, you step into a bygone era. The wind disperses into a cloud of dust which reveals a dark grotto wherein colourful reflections glimmer. You pick up a lamp at your feet and explore the darkness, admiring the walls covered with myriad illustrations. Gradually, the storylines begin to unravel before your eyes – literally and figuratively.

ZHAO Qi on Leaving for Baidi City in Early Morning…

Tang poetry is the cultural epitome of a prosperous dynasty. But what does the Tang Dynasty look like? It is hard to imagine, and even more difficult to restore. There are no paper versions of the landscape paintings of the Tang Dynasty surviving today. Our images are mainly based on the silk painting “Boat and Pavilion” by Li Sixun, and the painting “The Tang Emperor Travelling to Sichuan” by Li Zhaodao, as well as the Tang Dynasty murals in Dunhuang’s Mogao Grottos. These treasures enable us to imagine the elegance and grandeur of the Tang Dynasty.

This second VR experience in our trilogy allows users to literally enter the world of Tang painting, immersing themselves in the context of idyllic Tang poetry and imagery, adapted to virtual reality. We chose the seven-character Chinese quatrain “Leaving for Baidi City in Early Morning,” which was created by Li Bai – a romantic poet of the Tang Dynasty – during his return from a period of exile. After careful study of the Tang Dynasty paintings, we applied 3D technology to build digital models of the river and mountains – creating a three-dimensional interpretation of the reference materials, and immersing users in a picturesque environment.

Wearing the VR headset, you find yourself in a Tang painting and poem brought to three-dimensional life. Rowing the wooden boat, you pull yourself along the Yangtze River through the Three Gorges, amid cascading mountains echoed by ethereal bird chirps and monkey cries, with surprises around every bend. Within this environment, we designed a number of classical Chinese interactions, allowing people to experience the poet's journey and mood in a deeper manner than by reading. You have a visceral impression of Tang poetry in minutes.

ZHAO Qi on The Warriors…

The Terracotta Warriors of the Qin Dynasty are a large-scale underground army in an impressive array. Since their accidental exposure in Xian province in 1974, the Terracotta Warriors have been tirelessly excavated and restored for more than 40 years, unveiling part of the mausoleum of China’s famed Qin Emperor. Although the Warrior pits are restricted to the public, you can experience one of the Eight Wonders of the World up close and in person, through the world of VR.

Thousands of life-sized visages stand around you. Do you recognize any of them? You stand there as if among the 85 craftsmen of the Qin Dynasty who marked their names in history with the story of an empire, the details so vivid that you can make out hair and palm prints. The Warriors’ weapons – crossbows, swords, spears, axes, shields, horse-drawn carriages – are at the ready. Which will you choose? Who will you be? To what destiny are these heavily armed forces headed?

Independent filmmaker Wen FENG, a graduate of the Beijing Film Academy and former Director of Development for Disney China, discussed the ins and outs of creating The Cave, Impact Eyes’ award-winning first piece in the Walk in China trilogy.

Wen FENG on working with VR…

My background is in entertainment media. I worked in the animation industry for more than 10 years, mostly in development and production – including at Disney here in China. The Cave was my first VR project, and the first room-scale VR experience for our producer ZHAO Qi and most of the team. Everybody brought their own experiences from different backgrounds. Our writer researched the historical aspects of The Cave – most of the narration you hear is written by her – but our team had many meetings to determine the content. We thought about how people conceptualize their existence, and discussed that philosophical question a lot. But towards the end, we mainly focused on helping people intuitively navigate VR. Unfamiliar users need more help than you anticipate, and do things you may never imagine – often based on their expectations from traditional media.

Wen FENG on the differences between VR and traditional media…

A VR project is very different from traditional media. There were so many unexpected things that happened after we finished our initial development. Beta user interactions in VR led to lots of adjustments. The Cave is also different from a VR game. A game is very direct: you keep things entertaining so people will play. With our Chinese cultural works, the goal was for people to enjoy the experiences without feeling bored or lectured to – keeping them vivid and interactive – while not cheapening those experiences. The earliest version of The Cave offered immediate interaction, but most people were interested in looking at the cave itself, so we allowed time for them to do this. It’s a rare Chinese cultural treasure after all, and restricted to the public. We also allowed time simply for people to become familiar with the environment of the VR space.

Wen FENG on guiding the user in VR…

We spent much of our time thinking about ways we could make people feel involved, but not directed. The Cave’s “pop-up book” element is one example. We wanted people to watch the animated story of The 500 Bandits, and we hoped that people would stand in a certain spot so they can have the best point of view in VR. But we didn’t want say, “You go over there!” That’s why we created a huge table, which rises from the floor as a base for the book. Scale and sound are important components of VR. We’ve even seen people step back because they are afraid to be “bumped” by the table. This is an invisible way to guide the audience and direct their experience.

Wen FENG on adapting the Dunhuang cave paintings…

The most famous part of the Dunhuang caves is the paintings, which are essentially Chinese art history. It was important to adapt the paintings in a new way – to make them more interesting in the VR medium – but to keep the original meaning of the paintings intact. The simplest idea was to have the painted characters come to life on the wall, but that missed the spatial benefits of VR. So we had a brainstorm, and the “pop-up book” idea came about. It’s a format that almost every culture has some tradition of, and is used to introduce the Dunhuang cave paintings. We tried to apply the original painted imagery as directly as possible, with a limited animation style. I found different reference points from which to tell the paintings’ stories, established the staging of the animated characters, determined how high the picture book would be to provoke people to look up and generate feelings of respect, etc.

Wen FENG on the physical versus the spiritual in VR…

Conceptualizing the spirituality of The Cave took place from the start, while our screenwriter was researching the history of the Dunhuang caves – including all of the stories that were painted on the cave walls. There are so many, but we narrowed it down to three, and then my job was to figure out how to visualize them. We reviewed some existing VR games that are set in the Dunhuang caves. The digital assets in those games are “realistic”, extremely colorful and a little creepy. I wanted our fantasy elements to be more transparent and ethereal – in the same way that a Chinese painting can be suggestive by using negative space. When you first put on the VR headset, everything in The Cave has volume. If we then added ten fairies around you as solid digital characters, you’d feel crowded and perhaps alarmed. It wouldn’t be very beautiful and wouldn’t evoke the appropriate spiritual feeling. The scanned Dunhuang cave data is a representation of an actual space, and the animated ethereal visions act as a representation of a spiritual experience, such as the Dunhuang monks might have. So we have a “physical” space and a “mental” space in VR.

Wen FENG on transcendence…

The fairies of Cave 285 are your guides from the beginning, and we use a fantasy premise to reveal them. So when people have this experience they know they’re on a spiritual journey. In the pop-up book section of The Cave, we emphasize the last page, which leads the participants to look up into the glorious light of the Buddha. You later find a beanstalk in a little alcove, which is an actual place in the cave: ancient monks would meditate there. And that leads you toward a tree – not just any tree, but the Bodhi Tree. So, The Cave starts from a historical point of view, and then leads the audience to a state that corresponds to enlightenment - of being a Buddha. In the end, you can “become” Buddha.

Kevin is the author of AWN's Reality Bites blog, his musings on the art, technology and business of immersive media (AR, VR, MR) and AI. You can find Kevin's website at www.kevingeiger.com and he can be reached at holler@kevingeiger.com.