Bill Desowitz concludes our two-part coverage of Pirates of the Caribbean: Dead Mans Chest with a report on ILMs innovative Imocap system.
Tasked by director Gore Verbinski to come up with more complex and authentic-looking CG characters in Dead Mans Chest, since Davy Jones and the crew of The Flying Dutchman would be interacting closely with the live actors, Industrial Light & Magic put its R&D team to work on a new incarnation of its proprietary motion capture system, dubbed Imocap. The results of Jones are so impressive, in fact, that people have already begun talking about the sea-encrusted villain with his creepy tentacle beard as the next great CG performance breakthrough.
Weve done a lot of computer vision work here in R&D for the last several years and we were hoping to apply that to motion capture work outside of the MoCap studio some day, remarks Steve Sullivan, director of R&D at ILM. Dead Mans Chest provided an opportunity for [remote MoCap] and a clear case of [requiring] that same quality on set where we needed those actors together in a scene for those hero performances. So we worked with the production team to nail down constraints of what we could get away with and whats off-limits.
Imocap became a new protocol for measuring the actors and obtaining data during the actual shoot for the creation of skeletal motion in the computer. The software contained added functionality and new ways of tracking data. Special sensor-studded suits for the actors playing CG characters were created, which were more comfortable than typical MoCap outfits, as the actors were required to wear them in a variety of simple and treacherous conditions. On set, I wore a gray suit, which had reference points comprised of white bubbles and strips of black-and-white material, so that when they come to interpret your physical performance, theyre better placed to do so, adds Bill Nighy, who plays Davy Jones.
According to Sullivan, the suits needed to be dignified. They had to be comfortable and not look stupid. There were a few iterations of the material itself, which started out as a cotton blend but ended up being a stretchy, semi formfitting material. And we arrived at a neutral gray to help with our lighting calculations... and we used some markers and bands to help with the capture process itself. Those needed to be comfortable as well. Cameras were based on location and shooting conditions.
For shots where we used reference cameras, Kevin Wooley, our Imocap lead, housed some cameras in watertight enclosures and wired them to a computer for storing the images, explains animation supervisor Hal Hickel. This was great for the onset stuff. For beaches and jungles, we used untethered cameras with lightweight tripods. They were a little more trouble on the backend because they werent synchronized to each other, but both solutions worked well, and will continue to be used on the third Pirates movie [At Worlds End].
Thus, by integrating the MoCap process with the actual shoot providing the animators with hero plates with the actors in them, casting their real shadows and making good eye contact with the live actors they were able to create, for instance, a more expressive, nuanced performance out of the maniacal Davy Jones, with the help, of course, of Nighy.
We had new ways for the computer to analyze the images, Sullivan continues. The software piggy backed on MARS, the matchmoving [and tracking] solver. It understood what the actors could and couldnt do. Our process is more holistic than traditional MoCap. We try to capture the whole body at once from different kinds of information, and that allows the flexibility to use many kinds of cameras and to work with partial information sometimes.
The product of Imocap comes out as an animated skeleton, just like regular MoCap, and the animators do with that whatever they want, with artists in the middle running the post process. Sometimes theyll need to cheat the body to get a better composition of the image. But the advantage is that the animators are overriding things and animating for performance reasons rather than just getting the basic physics and timing down. That all comes from the actor.
Although ILM is currently developing its own facial performance capture system, Hickel determined this wasnt the time to introduce yet another R&D component. We have a lot of confidence in our facial animation, so we decided to do it by hand. The creature pipeline was being moved over to Zeno and most of the faces were different enough from the actors anyway.
Hickel adds that theres still a lot of animation artistry at work. The CG characters werent 1:1 proportional copies of the actors, so theres a lot of reinterpreting their motion and figuring out how to get a good performance out of a guy whos head is made of coral. We had a little more freedom with some of the background actors because their faces are so different, such as Ogilvy, whose head is basically a giant sea sponge and he has one eye in some weird orifice. Davy Jones is the most complex and human-looking CG character. Hes 100% CG even his eyes. We knew it would be difficult, but we figured we could get there pretty quickly. What was just as difficult was the whole spark of life. The thing about Bill was he wasnt a stone-faced villain. It was a very mercurial performance he was constantly changing his expression and delivery. Nobody expected it. Every scene wed stare at it and study it.
Concludes visual effects supervisor John Knoll: For us, its taken character animation another step forward with Davy Jones and how nice Bill Nighys performance comes through.
Bill Desowitz is editor of VFXWorld.
Bill Desowitz, former editor of VFXWorld, is currently the Crafts Editor of IndieWire.
