The scenes were shot on a 20x20-foot capture space that included a 16-foot tall vertical space to accommodate scenes with stairs, ladders, ropes, etc. The capture volume was covered by 200 infrared cameras. This volume was designed by CG supervisor Albert Hastings and capture supervisor Demian Gordon. For each performer, there would be 60-80 markers for the body, and between 40-70 markers for the face. Each set of facial markers were specifically placed on each performer, so that his/her particular range of motion and muscle structure was captured with best fidelity. By using 200 capture cameras, the crew was able to shoot with six performers at a time, capturing both face and body data simultaneously. The flexibility of the system allowed the actors to perform entire scenes without having to stop and do pick-ups.

“For the human characters, my estimate is that 75-90% of the body movement ended up being mocap, with 50-70% for the facial performances. The ratio was determined on a shot-by-shot basis. The eyes and the fingers were always keyframed, and since they are the most expressive parts of an animated performance, there was always a lot of input from the animators on each shot. Personally, I think there is always room to explore the ‘what ifs’ of a shot, as long as it fits the intent of the story and the personality of the character. At times, the intensity and sincerity provided by the actors did not translate 100% accurately onto the animation rig, mostly due to the fact that it was not an attempt to replicate the actor’s features. So the animation team often went in on top of the facial mocap data to make the desired emotion read more strongly on screen. There were also some shots where, for various reasons, we needed to add something that wasn’t captured on the live-action set. Whenever we were called upon to do keyframe animation, we strove to maintain a stylistic consistency in the way the characters moved. The motion capture process has a certain texture to the movement that is slightly different than most keyframed animation.”

 

Building an Animation Pipeline
Imageworks opted to build the facial animation controls on a two-tiered system. The first tier was based on a system that codes actors’ facial expressions into a library of poses based on the actions of certain muscle groups. The second tier was a proprietary keyframe tool called, Character Facial System, CFS. “We developed a way to blend these two tiers into a very flexible interface,” Hofstedt explains. “It was theorized that the raw motion capture data on the actors’ faces could trigger isolated facial muscle groups in combinations to convey expressions of emotion on the animated characters. The actors made faces into the camera for many different poses (i.e., Inner Brow Raiser, Cheek Puffer, Nasolabial Furrow, Lip Tightener, etc), plus an additional set of phoneme poses. There was not a generic batch of shapes forced upon each character. Each individual actor made their version of the poses and the animation team interpreted what they did and applied it to the physics of the animation rig. The individuality of this process helped make each character unique. This interpretive process is especially what made our process on Monster House different from Polar Express. Our characters did not have to exactly match the likeness of any well-recognized performer.

“The chosen frame for each of the facial expression shapes performed by the actors on video was given to the animation team who used the CFS system to create the individual facial poses. These poses were saved and named under a custom naming convention devised by the facial integration team. After a full set of facial expression shapes and phoneme poses were created and stored, they were given back to the integration team to run the facial mocap data curves. In the early stages of testing the process, there was a lot of back and forth between the facial expression shapes team and the animation team. Sometimes, we’d have to adjust the range of the actual shape we had created (i.e., making the Lip Stretcher pose narrower or wider as needed), or maybe the facial expression shapes team determined that they had to adjust the percentage of influence of any given shape (i.e., amp up the influence of the Brow Lowerer). These choices were built into the facial expression shapes poses that would be driven by the mocap data. Once we settled on the facial expression shapes pose library, and determined that we needed the flexibility to go even further, we could concentrate on ways to push the performances to a new level.”