Google Spotlight Stories Debuts ‘Rain or Shine’

Navigating the uncharted waters of 360 immersive VR entertainment with innovative and charming films like Duet, On Ice and Pearl, the Google ATAP team has once again given audiences a reason to don a headset and stumble about their living rooms with smiles on their faces and springs in their step. They’ve just debuted their latest interactive Spotlight Story, Rain or Shine, produced in collaboration with London-based Nexus Studios. Led by executive producer Karen Dufilho, producer of Oscar-winning shorts Geri’s Game and For the Birds among others, and creative director Jan Pinkava, director of the Oscar-winning Geri’s Game and Windy Day, the first Spotlight Story, the Google Spotlight Stories group continues to explore new ways to tell interactive stories in the VR space while expanding the creative and technological methodologies and toolsets needed to move animated storytelling from the fixed screen to the virtual world.

I had a chance to watch the film on Google Cardboard as well as talk to the film’s director, Felix Massie and executive creative director and Nexus co-founder Christopher O’Reilly, this past June in Annecy. With the film’s debut, AWN can now share that conversation about the challenges and strategies of creating immersive 360 interactive stories.  Make sure to watch the film below on YouTube 360 - it’s also available on the Google Spotlight Stories app on iOS and Android phones. Also, check out the great behind the scenes video about the making of the film, as well as YouTube’s How to Watch 360, which features Rain or Shine. 

Dan Sarto: I had a chance to watch Rain or Shine yesterday while I danced around the room wearing the Google Cardboard headset. Forget for the moment that your film was a full 360 experience. Starting with the really crisp and stylized character designs, the overall visual style and colors, both the visual and story experience were excellent. How did you guys hook up with the Google Spotlight Story folks? How did the project come about?

Christopher O’Reilly: It was about two years ago, actually exactly two years ago, in Annecy, when we met with Jan Pinkava [Google Spotlight Stories creative director and Oscar-winning director of the animated Pixar short film Geri’s Game]. By that time, I think he had done Windy Day and Buggy Night [first two Google Spotlight Stories].

They were just beginning to think about working with external studios for the first time. Jan knew our work. We’d received an Oscar nomination in 2009 for our animated short This Way Up. We'd done commercial work for Chipotle that he knew and also things like the title sequence from Catch Me If You Can. So it was from short form stuff he knew us in London. He was interested in meeting studios that might get like to work with Google. He gave us a brief of the project. At the same time, Nexus had been exploring interactive storytelling quite a lot. We're quite unusual in that we've got coders in our studio and everyone has been engaged with interactive stories for about four or five years. We were excited about the brief because as well as storytelling obviously, it also talked a little bit about interaction design and that was really where we started exploring.

We have a great roster of directors who work with us in the studio -- Felix is one of them. The creative brief from Google called on us to explore, get familiar with the technology and come up with new ideas. We generated probably ten to 15 different ideas initially. That was an interesting process, directors trying to get their head around this entirely different way of telling a story.

And actually, the more we explored, the more we learned how different telling a story in this space is. Some ideas felt beautiful but not particularly suited to 360. Some felt like they were really embracing the possibilities of what you could do, not just in terms of 360 and looking around, but also in terms of a story that's being told in real-time on your phone, which is actually one of the most important features of what's going on in Spotlight. It's not 360 that helps grab the emotion, it's the real-time action that creates that sense of magic. And interactivity.

Felix was rather prolific with ideas -- he had quite a few good ones. Over a short period, we developed a few to run alongside each other and Rain or Shine came out as the favored route.

DS: Creatively, how did you bridge the gap between your established thinking on linear fixed-screen storytelling and 360 immersive experiences in a way that excited Google enough to greenlight the project?

FM: This project was quite different from the things I've done before -- I've done dialogue-driven stories before so this I knew had to be a silent film. The three [Google Spotlight Stories] we saw that were finished prior to our pitching were Windy Day, Buggy Night and Glen Keane’s Duet. They're all brilliant, but I think Buggy Night was the one that got me because like my films, it may be a bit cruel to the characters sometimes. Buggy Night is about trying to shine a light on some bugs that are trying to escape from a frog. Every time you catch them the frog eats one of them.

It's almost a simple game. The narrative is simple, but really pleasant to experience. That was what I liked about what you could do with the technology. The idea of Rain or Shine was always to spoil the girl’s day just by the very fact that you're watching the film. You're making the situation worse for her in the film -- if she runs to the side the cloud will eventually move over to the side, but as she moves the clouds move more quickly. That's quite a subtle trick in the realm of a film.

CO: It gives you a degree of agency, doesn't it?

FM: Yeah. That was what appealed to me and that's what Google liked about our idea, that it was a kind of...meta, is that the right word?...that you're having an effect on the film.

DS: Sure. So how did you start to learn their VR toolset? How did you begin to approach this project, working with brand new 360 technology that was being developed while you were using it?

CO: There are two areas [to consider]. One obviously is the technical area and the other is the creative process. Technically, we were blessed with a very strong team that had already explored some of the ideas of filming and interactivity. We had a great relationship with the Spotlight team because at this point everything's really Beta with them…pre-Alpha really. We're going back and forth so there's a lot of we encounter a problem, overnight they provide a solution for when we get up in the morning. There was no off-the-shelf toolset.

DS: You're constantly going back and forth with Google on this.

CO: That was one of the nicest parts from the technical side, an amazing privilege. This thing being developed, we were one of the first people outside of Google to work on it with them -- at that point, the other films had been made at Google. So it was really interesting, that technical dialogue.

DS: How did you storyboard this project? How did you plot and block out the story and the interactivity?

FM: We wrote out the script as a sort of “This is what's happening to the main character” thing. I don't know if Chris can find it on his laptop, but it was…

CO: I have to show it…

FM: We had this grid. As you can imagine, a linear film is a straight line. He's going to pull it up…good, there you go. So [explaining grid on screen] this is time along the top and these are locations going down.

CO: In a linear film, it would all go across there [points to a straight line across the top], but our golden thread as we call it, which is this yellow line, is the main character Ella. We had these gaps. Initially that was very complicated. We had all sorts of ideas to fill them, like a postman having an affair and stuff like that going on around the corner. But then we found out that actually, sometimes you need to scale back the side events because they might become a distraction from what the main story's about. Hopefully the things that are in the world are there to enhance it and make it feel more immersive rather than be a huge distraction.

FM: We've got things like if you look at a train bridge a train goes by. You look up at the church the bell just goes “Ding.” It's people outside of a pub, stuff like that. It's just nice little touches to make it feel more alive.

CO: The thing is, you have to see this golden thread, in the order these items are presented. But if you wander off in-between, you can always come back.

DS: You can come back to that central narrative track.

CO: It was interesting because we spent a bit of time right at the beginning, speaking to people who worked in immersive theater, who did experiential theater. Because we were thinking, “Well they don't know where people are when they tell a story.”

But those stories can have emotional impact. We spoke to people like Punchdrunk [a theatre group], to people with a background in that type of storytelling. We spoke to people in the games industry about how they corralled people within the playing space. We even spoke to architects who do things like theme park design and airports. They actually know how to subtly help you make choices.

FM: Based on how things present themselves.

CO: You have free will, but you're also more likely not to be following a certain direction. So, you know where people are.

DS: All told, how long did the project take to complete?

FM: There were different stages. I pitched before Christmas [2014], then we had a meeting in the new year. Then we sort of got the “Yeah, they’re going ahead with us.” Then there was a proof of concept phase, which is where we had to show that we could technically make what we were proposing. Then we got the greenlight for that. Then we delivered something before Christmas again that year [2015], but then we had to do the sound and other final stuff. That was another few weeks. Then we were just tweaking little bits and pieces.

DS: So well over a year?

CO: Yeah.

FM: A year and a few months.

CO: Some of that, a big chunk of that, was driven by that back and forth we were having with Google. And in a sense, that was a big part of the process.

DS: Start, stop, start, stop, start stop.

CO: The proof of concept was just the proof of concept. We didn't use any of that content for the final film.

DS: What were the biggest challenges you faced on this project?

FM: One of them definitely was the shades of the strip of rain. At the beginning of the project, that was impossible. That was a technological advancement that Google was quite proud of.

CO: In real-time, that’s affecting the environment, changing the color of things, moving with you through the space. Effectively that makes for a quite complex piece of interaction design.

DS: It's got to render in real-time.

FM: Which is 60 frames per second.

CO: If you're from the film community and technically minded, you understand that. But a lot of people who pick it up don't understand quite how complex that process is. And that that's all happening on your mobile phone. It's a kind of a miracle.

DS: More than any of us properly appreciate.

FM: You know that yellow line in the grid? There are actually three of them. There's a camera that has its own story line. The cloud has its own story line and Ella has her own story line. They all power themselves -- they all have to meet up. But, they can meander off on their own course for a moment. That was something technical director Mark Davis, even though he didn't think would be possible at the beginning, knew that we wanted, where certain things pulled us and certain things didn’t, carrying on without the main story going on. That was quite a eureka moment. Getting that working was a good thing because it opened up the possibility that this could happen or not happen. For example, the cat and the pigeon. If you watch the cat chase the pigeon the pigeon will land on the ice cream van. If you don't watch the cat and the pigeon the pigeon will never land on the ice cream van, but the story goes on without it. This is what we mean by threads of stories coming together. That was quite a complicated, good thing to achieve.

CO: Speaking to Rachid El Guerrab yesterday [Google Spotlight Stories technical project lead], he was saying that this is by far the most complex Spotlight Story so far in terms of the level of interaction and real-time differences in terms of how one person might experience it from another. Also, in terms of technical complexity and working across all the platforms it’s going to be delivered on.

DS: From a visual design style, the film is quite engaging to look at. The town environment was bright, without too much distracting color. The character designs were quite cute, very cartoony but stylized. There was a fine, minimalist edge to all the designs. Tell me a little bit about your decisions with regards to what this film actually looks like.

FM: I guess there are two main things. One is I've always worked in 2D. Robin Davies, our character designer, is fantastic. I really love his style and how it can work in scales. It can work close up -- it can work far away. It's clear posing and it's got the 2D sensibilities. When Google approached us, they said they wanted a film, not necessarily a VR gaming experience. They wanted something that felt like fun. This film achieves a cinematic 2D animation sensibility while feeling a bit more modern. The second one is that it has to run real-time on the phones. So we knew we had limits. We couldn't have fully rendered fur, for example, not that Ella would have fur, but you know what I mean. Robin's style is perfect for that. Really simple geometric shapes, things like that. Films I’ve done before have been stickman style.

Though this is more detailed than that, it didn't feel like too much of a departure from the kind of world that I would make. It’s a really pretty design I think.

DS: Last thing. With a project like this under your belt, what does it mean for you as a studio?  Everyone's diving into VR, whatever that means, and a lot of it is fueled by tons of R&D money. Setting aside games, for artists used to creating narrative content, mostly they’re getting paid to create a particular piece of VR content, rather than making money by creating entertainment products that are sold through one or more commerce-driven channels. Where do you see your future in VR?

CO: Some of the creative processes we use involve a universal VR toolset, which has been really helpful. Technically, Google has been showing how they can take this across different platforms. In terms of where the business of VR is going, it’s still very much a proto field. We're now developing other original content ideas in VR. I'm looking at quite novel ways of how we can fund those and get those out. There's definitely a demand for content.

But we are in an unusual period. We have to play with the landscape as it is and it is still much more about launching the headsets and getting them out to an audience and making people understand there can be something valuable and emotional when you use these things.

DS: Right. In the agency world, do you see VR creatives fueling the commercial business?

CO: Brands, yeah.

DS: Do you see the number of discussion's picking up with regards to potential projects in that area?

CO: Absolutely. There's a lot going on in that space. Remember, VR isn't one thing. It's a series of very different platforms, from the quite complex ones like VIVE through to YouTube 360 and Cardboard. Agencies are starting to wrap their head around those and use them in different ways depending on whether they want big audiences or big experiences at an event.  But there's definitely a big take up from brands. We’re doing more work in that area.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.