MPC VFX supervisor describes the studio’s innovative efforts handling 1200 shots alongside Rob Legato on Disney and Jon Favreau’s blockbuster family adventure hit.
Much like an explorer blazing trails and charting new ground on a major expedition, MPC VFX supervisor Adam Valdez spent two long years and encountered numerous creative and technical challenges during his journey working alongside filmmaker Jon Favreau, VFX supervisor Rob Legato, and animation director Andy Jones in the innovative and widely acclaimed reimagining of The Jungle Book. And just like a pioneering explorer, Valdez wasn’t exactly sure what new territory he’d uncovered until well into the journey. He notes, “Every time you do a movie you don’t know if it’s going to be any good. You just do your part. You’re working away on it so much that you lose perspective after a while. But on this film, it was at the three quarters stretch that we started to feel that something cool was coming together.”
MPC London handled the bulk of the movie’s VFX, some 1200 shots, while Weta Digital was responsible for the King Louie Sequence. MPC’s Bangalore, India facility was also involved, handling a range of tasks including asset creation, animation, lighting and compositing. Industry veteran and two-time Oscar-winner Rob Legato was the film’s overall VFX supervisor as well as 2nd unit director and was instrumental in how the film “looked” and was shot. According to Valdez, “Rob acted like this eye for what we would get if we really shot this live. He helped us to do the lighting and camera work in the most naturalistic way and advised in terms of colour and composition.”
Valdez and MPC came onto the film knowing full well how uniquely challenging the production would be. “When they came to us wanting to do the movie with bluescreen and one kid, it was daunting,” admits Valdez, who also had to deal with the principal photography being shot in native 3D. “When you combine lots of different source materials, it gets difficult bringing images together at the compositing stage to keep the look clean and photographic. We made a decision early on that we would do mostly geometry-based environments. That’s a big decision because it takes a lot of manpower. You have the ray tracer [which accurately portrays how light bounces around] rendering the animals and their environments together which unifies the look. You also get incredible stereo resolution because every single blade of grass and leaf is in stereo.”
The adoption of RenderMan RIS required considerable changes in MPC’s pipeline. “That [adoption of RenderMan RIS] required us to write all brand new shaders for every object,” Valdez explains. “We also had to convert a lot of our lighting toolsets to work with the new renderer. We brought in a Houdini engine and did a lot of fur and water simulation on a larger scale than we had ever done before. We did a lot of pipeline modifications to handle the scale of the sets. We had scattered tools developed within the set department that allowed us to place plants, dead leaves and twigs all over the ground.”
Valdez’s MPC team setup shop alongside the film’s main production crew in a Los Angeles warehouse, where the entire film was shot. They were soon immersed in integrating previs and virtual production efforts used to plan the various shots. “When a team from MPC London went to Los Angeles, one of the key things we hooked into was how Jon Favreau was developing the movie as an animated feature,” states Valdez. “He had a storyboard department and his editor on from the beginning. They were working the story reels trying to refine the movie. There was a massive virtual production effort using motion-capture. Our team in Los Angeles went through all of that previs so that we could go on-set with a shoot plan for every sequence, working with the DP Bill Pope and gaffer Bob Finley to make sure we had lighting of Mowgli that would line-up. Audrey Ferrara, who is our environment supervisor at MPC, worked with production designer Chris Glass to take concept art and turn it into digital sculptures in ZBrush and Maya.”
“Once animation did a blocking pass on a sequence, we asked Jon to approve it,” Valdez continues. “It didn’t mean that the sequence was locked down but departments like effects, sets and technical animation could start to engage in a more serious way and break it down into what we had to do.” Even the film’s sole live-action element was occasionally replaced. “There were a couple of cases where we replaced the kid with a digital double. The edit changed, so we didn’t have the shot needed from the photography. We had to make some stuff up out of thin air,” notes Valdez.
Creation of environmental sets was a huge challenge. “Audrey Ferrara got a design pack from Chris Glass that was a visual rule book of every section of the jungle, such as the kinds of trees you could use and the colour pallet he liked for a particular area,” explains Valdez. “Audrey did a breakdown of all the locations in India and we sent a team out from our Bangalore facility to do an extensive shoot out there. We visited over 50 locations around the country and brought back hundreds of thousands of photographs. These were panoramas, close-ups, textural details, and highly detailed coverage of individual objects like trees and rocks on which you could base models. It was a huge reference library but also provided the raw materials we used to build our assets and world. In this film we didn’t use a lot of straight matte paintings - we used a lot of textural projections and model making from that photo shoot. We laid out all of the individual plants.”
The team did an element shoot, using assets in various ways. According to Valdez, “We developed ways of creating element libraries digitally. For example, in this movie there are a lot of bugs and pollen floating in the air. Some scenes require us to simulate and render those directly in the shot, but you can also render them to a card that is placed in our Nuke compositing software at different depths within the scene. It’s always a mixture for every scene.” Some creative interpretation was required. “Whenever you see any reference of an Indian jungle, the air is full of dust, bugs, plant seeds, dead leaves falling down from the canopies, and heat plumes where you see tiny particulates rising,” says Valdez. “The problem is if you do it too accurately you end up with something all too distracting. As the compositing is coming together you might say, ‘The shot needs some depth so let’s get some movement in the back and put a few bugs back there to make it feel more real.’”
A wide and diverse set of animal characters populates scenery throughout the film. MPC artists and reference photographers were able to go out and capture a large amount of animal reference footage. “The guys in the rigging department spent many months developing and researching facial anatomy on big cats, wolves and bears, as well as how they move. Ben Jones, our character supervisor, developed a whole bunch of muscle and skin simulation techniques for this movie that up the naturalism - they provide the animators the control they need to get the right motion.”
No motion-capture was used on animal animation. It was based off of Internet video reference and clips from the MPC library that allowed the animators to line-up action to action with whatever their character had to do in a scene. “We have a scene where Mowgli is riding down the river on the bear’s stomach,” explains Valdez. “A fur animator worked with an effects water simulation artist, going back and forth until the fur and water looked connected. When it comes to Baloo charging through plants and grasses, the technical animation group would do the fur simulation, move it through the grasses, then rig different plants and branches in order to make sure that every time he touches or brushes past them there was a natural movement.”
Throughout the film, Mowgli has conversations with the various animals he encounters, which are voiced by Ben Kingsley, Scarlett Johansson, Bill Murray, Idris Elba, Christopher Walken and Lupita Nyong’o. “We made sure that the facial rigs themselves were based on a mixture of shapes that were required to hit human sounds but also mimicked what real animals do with their mouths and faces,” states Valdez. “The animators work out a tool kit. This is what the real creature would do, these are the muscles they have that work on their face, and I’m going to dress in a little bit of a helper shape around the mouth that makes me see that as a ‘m’, ‘p’ ‘t’ or ‘v’. A lot of what you learn with lip work is reduction. When you watch real people talk they don’t articulate every single shape and sound. Some of the big broad fricatives and some of the key vowel shapes or tongue shapes are required and the brain will believe that animal just made that word.”
The director had rules about how beautiful MPC was allowed to make scenes. Notes Valdez, “Jon didn’t want it to feel like every scene was the most spectacular version, but rather we’d gone out in the morning, shot it and that was the best we could do. We made sure there wasn’t too much perfect continuity by varying the sky, camera angles, and the time of day. We would take two or three key angles from the sequence and use different skies and do lighting passes and colour grades mocked up in Nuke and Photoshop. The stills would be shown to Jon and Rob to get their reaction. Often matte painters would take the skies and produce a dome for the scene that helped show the line-up for every camera in the sequence that could help guide compositors and lighters later on. That was critical because it established the relationship between the Mowgli photography and the rest of the scene.”
One of the most important criteria needed for crafting scenes was determining if an audience would believe that Mowgli was actually standing in that place talking to that animal. Careful planning was key. “All the pre-production planning helped, making sure that we knew where all of characters were, what they were doing and where we should be looking,” explains Valdez. “Jon had this brilliant idea of using Jim Henson Company puppeteers to work with Neel Sethi, the actor who plays Mowgli, so that he always had a moving and engaging puppet. A young person leaves that to the imagination and it came through.”
Looking back on the project, Valdez pays tribute to his MPC team, as well as shares his personal fondness for a particular character in the film. “I’ve been on holiday for about five weeks to recharge my batteries because it has been a huge project,” he states, “One of the most gratifying things about doing this work is that you have so many good people working with you, from animation, sets, technology, software, lighting, compositing, and production. You have to be a marathon runner in a sense. We had over 100 compositors on The Jungle Book generating final images. This was a show that every single department had to bring their best and innovate.” He concludes, “Personally, I think the most engaging sequences are in the middle with Baloo. It’s the point where The Jungle Book slows down and lets you watch him interact with these animal characters. It’s the wish-fulfilment that makes the movie work.”