Davy Jones and the crew of The Flying Dutchman interact closely with live actors throughout the film.  

Thus, by integrating the MoCap process with the actual shoot — providing the animators with hero plates with the actors in them, casting their real shadows and making good eye contact with the live actors — they were able to create, for instance, a more expressive, nuanced performance out of the maniacal Davy Jones, with the help, of course, of Nighy.

“We had new ways for the computer to analyze the images,” Sullivan continues. “The software piggy backed on MARS, the matchmoving [and tracking] solver. It understood what the actors could and couldn’t do. Our process is more holistic than traditional MoCap. We try to capture the whole body at once from different kinds of information, and that allows the flexibility to use many kinds of cameras and to work with partial information sometimes.

“The product of Imocap comes out as an animated skeleton, just like regular MoCap, and the animators do with that whatever they want, with artists in the middle running the post process. Sometimes they’ll need to cheat the body to get a better composition of the image. But the advantage is that the animators are overriding things and animating for performance reasons rather than just getting the basic physics and timing down. That all comes from the actor.”

Although ILM is currently developing its own facial performance capture system, Hickel determined this wasn’t the time to introduce yet another R&D component. “We have a lot of confidence in our facial animation, so we decided to do it by hand. The creature pipeline was being moved over to Zeno and most of the faces were different enough from the actors anyway.”

During the actual shoot on beaches and jungles, ILM used untethered cameras with lightweight tripods to measure the actors and obtain data that was then used to create the characters’ skeletal motions in the computer.  

Hickel adds that there’s still a lot of animation artistry at work. “The CG characters weren’t 1:1 proportional copies of the actors, so there’s a lot of reinterpreting their motion and figuring out how to get a good performance out of a guy who’s head is made of coral. We had a little more freedom with some of the background actors because their faces are so different, such as Ogilvy, whose head is basically a giant sea sponge and he has one eye in some weird orifice. Davy Jones is the most complex and human-looking CG character. He’s 100% CG — even his eyes. We knew it would be difficult, but we figured we could get there pretty quickly. What was just as difficult was the whole spark of life. The thing about Bill was he wasn’t a stone-faced villain. It was a very mercurial performance — he was constantly changing his expression and delivery. Nobody expected it. Every scene we’d stare at it and study it.”

Concludes visual effects supervisor John Knoll: “For us, it’s taken character animation another step forward with Davy Jones and how nice Bill Nighy’s performance comes through.”

Bill Desowitz is editor of VFXWorld.