Oscar 2012: Joe Letteri Talks 'Rise of the Planet of the Apes'

The four-time Oscar winner talks about the VFX front runner.

Check out the Rise of the Planet of the Apes trailers and clips at AWNtv!

Weta Digital raised its performance capture and animation game for the Rise of the Planet of the Apes reboot. It's turned out to be a brilliant success and everyone is raving about the CG Caesar, performed by Andy Serkis. Senior visual effects supervisor Joe Letteri discussed the breakthrough.

Bill Desowitz:What do you make of this year's nominees?

Joe Letteri: The character-driven films were obviously rewarded but Hugo and Potter are complete inventions, giving the directors a lot of creative freedom.

BD: And obviously Apes leads the pack of character-driven films, thanks in large part to Andy Serkis, who is helping educate Academy voters more about the process.

JL:

People have been impressed by Andy's performance. He allowed them to connect with Caesar. The story now has a lot of possibilities to go in different directions. Here you can easily see the transfer of going to both worlds with the onset capability of performance capture.

BD: What has been the big take away from Apes?

JL: The rigging becomes part of the normal rigging process. The more people do this, the more the cost goes down.

BD:Let's go over some of the improvements once more.

JL:

We rewrote skin, muscles, fur, and eyes one more time to do them a little bit better. But I think making the performance look as realistic as possible is still the main thing that we accomplished. Unlike Avatar, we placed the performance capture actors out on location or on set with the other actors. Rather than using reflective optical markers for motion tracking, we developed an active LED system with infrared lighting that allowed us to work in a variety of conditions and match the cinematography. We also developed a new facial muscle system still in progress that delivers better capture and animation, particularly for secondary motion. It's a problem that's not easily understood because the facial muscles don't behave like the other muscles in the body. They are not so bound by the skeleton. A few of them are like on the jaw. But on a face they're moving other muscles around and other tissue, and there are deep embedded layers that have an impact on what kind of shape they do, which is really complex and why in the end we wind up sculpting a lot of these things.

BD: What about the fur system?

JL: We still use Barbershop. But we came up with a new system, so rather than growing fur procedurally, we decided to have the artist groom every fur directly. It's much more natural.

BD: What was the overall strategy with Caesar?

JL: We made Caesar more human because we wanted him to look a little more intelligent than the rest of the apes and to stand out among them. There's not enough time in the story to show physical transition, so it went into his design from the beginning. You could see it in his eyes: we made the irises a little smaller so you get a better idea where he's looking; the muzzle is slightly smaller; and the forehead is shaped a little bit more like a human's.

We made a new model that more realistically captures movement in and around the eyes and how they are affected by different lighting conditions. One of the drawbacks of doing performance capture in general is you've got that light on the face, which happens to flatten out the characteristics. In this case, though, because we were on the real set, we at least had the lighting to play off of Andy's face. But you still have to pay attention to whether it reads in the current lighting situation. You might not be getting the light in Caesar's eye, so you make slight adjustments to get a better read.

BD: The performance capture is so successful. What about the importance of the animators?

JL: Performance capture is still more artistic than mechanical, and the animators were more empowered to make creative choices. When you're capturing the shapes of the face, nothing on the face is ever fixed; there's nothing locked down to refer to it, so the first thing you have to do is figure out your baseline. And then you just look at it side by side with the performance from the actor and say, 'Does that look like the right performance or not?' If not, why not? Sometimes there are errors that you can fix; sometimes it really just comes down to interpretation.

BD: But the performance has to be believable.

JL: Right. He might be digitally rendered, but there's a soul when you look into those eyes,

Bill Desowitz is former senior editor of AWN and editor of VFXWorld. He has a new blog, Immersed in Movies (www.billdesowitz.com), and is currently writing a book about the evolution of James Bond from Connery to Craig, scheduled for publication this year, which is the 50th anniversary of the franchise.