Jack climbs the mighty beanstalk in Jack the Giant Killer. All images courtesy of Warner Bros. Pictures.

Director Bryan Singer not only got more kid-friendly with Jack the Giant Slayer (riffing on The Princess Bride and Clash of the Titans), but he also fully embraced virtual production and 3-D for the first time. Indeed, he played in the volume during the mocap sessions right along with the actors: choreographing the action and making selects right on through principal photography using the Simul-Cam system for integrating live action and virtual components.

VFX vet Hoyt Yeatman (The Abyss) oversaw Jack as the visual effects production supervisor. He and the filmmakers worked with Digital Domain and Giant Studios, which collaborated on the virtual production process. Meanwhile, The Third Floor provided previs of action sequences and MPC created the crucial CG beanstalk, sharing assets with DD when necessary.

"When we started three years ago, we talked about Avatar-like characters but this would be in real environments and Bryan wanted athletic and smart giants," Yeatman explains. "When the idea of Simul-Cam was first proposed it was an optical tracking system, but we shot Jack on location and very often in historical locales such as a cathedral that had restrictions for how you could shoot inside. We ended up shaft encoding, which came from the old days of motion control, and that worked really well and since the 3-D camera rigs [the movie was shot natively on Red Epic cameras] almost always lived on the cranes. It was a perfect platform to work on, so the guys at Giant Studios wrote some code and built a virtual model of the cranes accurately used for geometry reasons. So we were able to Simul-Cam anytime we wanted into those scenes and we brought Giant Studios out to the UK and built a mocap stage at Shepperton Studios."

Fye

It was critical for the filmmakers to have an accurate understanding of the 20-to-24-foot-tall CG giants' performances, position and timing while shooting scenes where giants drove the primary action and interacted with live actors; as well as for all-CG or mostly-CG sequences featuring giants. As Fallon, the two-headed giant, was performed by two actors simultaneously, Bill Nighy and John Kassir, the use of virtual production was particularly important in that it enabled both of them to pre-visualize the final product and base their performances around this unique character trait.

In a mocap shoot prior to principal photography, sequences driven by giants were choreographed and directed by Singer; Giant Studios capturing all body performance and Digital Domain’s virtual production team capturing all facial performance (using four mounted face cameras), simultaneously as a single, cohesive performance, in real-time in 3-D.

Fumm (center)

Digital Domain’s virtual production team then created "Kabuki": projecting video from its face cameras onto the giants’ characters so Singer and editorial could view the actors’ performance quickly and cost-effectively within the real time version of the performance captured. Digital Domain delivered the Kabuki asset to Giant Studios where it was integrated it into the real-time scene, which could then be viewed in MotionBuilder or played out as QuickTime files for editorial. This process enabled Singer and the editorial team to make performance selections from all options gathered via real-time render of the body performance on set plus witness camera material from the face cameras. To ensure all motions maintain the weight & scale of giants, most motions were slowed down by 15-20%.

Those cohesive giant performances were then used for on-set references during principal photography where they were viewed by Singer, cinematographer Tom Sigel and VFX supervisors through the Simul-Cam or viewed as QuickTime references, so they could see the giants’ performance, scaled correctly (at a factor of four) alongside live actors, within the CG environment. This additionally created great time and cost efficiencies in the post-visualization stage as Digital Domain was able to respond quickly to directorial and editorial changes in giants’ position, timing, performance and eye lines.

Fallon

"We had to raise the bar for virtual production for these actors to become these characters," remarks Stephen Rosenbaum, who came over to Digital Domain from Weta Digital after working on the Oscar-winning Avatar. "And that was huge because when you see Fallon delivering dialogue or emoting, you pick up Bill Nighy's expressions and mannerisms that come through in the character. And it gave the editors a lot to work with as well early on in the production. They didn't have to wait months for us to develop the blocked animation and then get down into dialogue later and ultimately everyone wondering whether or not the performance was going to come through."