Avatar: The Game Changer
For the animation, Weta put a lot of effort into the facial solves and tracking "because one of the problems with the way that we were doing it was you've only got a single point of view using one camera," Letteri continues. "Ideally, from a technical point of view, it would've been good to go with two or more cameras. But from a performance point of view, that was going to add weight, it was going to slow down the process to changing out drives and it was going to be cumbersome for the actors."
Weta also created a new optical solver for the eyes to track them and paid a lot of attention in animation to the movement to compensate for what the solver couldn't achieve.
FACS (the Facial Animation Coding system developed by Paul Ekman) was utilized once again by Weta. But one of the problems with FACS is that it doesn't cover dialogue. And so that came as a secondary layer, where the motion editors and animators looked at the data coming in and had to figure out what the track was doing and how to solve that. It's really hard to track the shape of a lip because it just changes constantly. So the system was built as a big solver that you could input training data into so that the facial editors would try and interpret what was going on and keep adding to the system until it converges on the right answer, and then have the animators go through it again and take another pass with the rig and make sure that everything behaved properly and worked in the right combination.
In building this whole world, Weta had hoped to at least create the plants procedurally, but ended up hand painting everything to make sure that it was of the highest quality and uniform in 3-D space.
They additionally adopted a global illumination system for lighting. "We came up with a system based primarily on image-based lights but then converted the whole system to spherical harmonics," Letteri explains. "What that meant was we could pre-compute all the lighting contributions in a scene and then put the characters and everything in with the lighting and the TDs could move the lights around. It would tell you what the influence was of all the objects around each other. And you could solve that in a global sense."
A new full-on compositing system was devised as well for 3-D. "We started outputting all the depth information for everything we were rendering, so you don't need to rely on mattes anymore," Letteri adds. "You know where everything is in space and can figure out the relationships. That's really important for things like the jungle, where you've got lots of plants and you could layer them in the right order based on depth because you're dealing in pixel to pixel to pixel. That will become standard for compositing from here on out because of the flexibility, even if you're doing a non-stereo movie. It's just easier to composite in 3-D than in 2-D."
Yet Weta still required vfx assistance and received great collaboration from ILM, Framestore, Hydraulx, Pixel Liberation Front, Blur Studio, Buf Compagne, Hybride, Prime Focus, Halon, The Third Floor, among others.