Isaac Kerlow looks at recent human 3D character animation developments in The Incredibles and The Polar Express.
The Incredibles and The Polar Express are two recent animated features that achieve remarkable creative results using different styles of human character animation. The Incredibles is an example of first-class keyframe cartoon character computer animation that integrates 2D traditional styles with the 3D computer style that we have come to expect from Pixar projects. The Polar Express offers an innovative approach that animates computer-generated virtual characters by applying realtime human performances and keyframe touch-ups.
Comedy and Action Through Squash-and-Stretch
Two aspects of The Incredibles represent significant departures from the topic and style of earlier Pixar movies: the human characters are central to the storyline and they are animated with considerable squash-and-stretch. To make the later possible the technical character team and the animators joined forces to develop new tools for animating squash and stretch humans in real time. The design and ease of use of the character setup rigs was driven by the animators' needs, and keeping the overall look-and-feel similar to earlier versions of Pixar animation software was an important consideration.
One of the main characteristics of the software tools used to animate The Incredibles was the layering of two key stages in the animation process: the bone and muscle calculations, and the squash-and-stretch system. Without the layering of these two stages it would have been difficult to provide animators with realtime feedback. In simple terms, the first stage of this layered process started by running the bone and muscle calculations through all the character's positions in a shot. Using statistical analysis the software determined the most significant changes in the character's skin throughout the shot, and "baked" those deformations into the model so that they could effectively be used as blend shapes. (Without the baking animated models played back at rates of eight to 10 frames per second.) After doing the first pass animation the animators saw the baked geometry and did not have to deal directly with the bone and muscle system. The second stage of the layered animation process consisted of applying the squash-and-stretch to the baked geometry as a post-process, and animators were able to visualize this in realtime.
Geometry decimation was another technique used to keep the playback as close as possible to 24 frames per second. Animators were involved in the process of hand-crafted decimation that brought some meshes to about 25% of the full geometry but kept, for example, the full and decimated silhouette shapes of each character identical. Each character had a decimated version, with most of the savings occurring in the body geometry and not the face. Decimated models did everything that full models did, and the deformation hierarchies remained the same but with much less overhead. Shots got finalized during the animation stage using the decimated version but the final rendering used the full geometry.
The approach to facial animation in The Incredibles followed the Pixar tradition of allowing animators direct control of all parts of the face. Facial animation was done with a multitude of deformers tied to macro-controllers, and without using blend shapes. Compared to the facial animation tools used in Finding Nemo these had more and better features, nothing groundbreaking but many incremental advances, for example: a greater number of controls to allow for squash-and-stretch, a new eyebrow package with spline-based controls, and the ability to do wild cartoony distortions of the face including the eyeballs something that Pixar animators traditionally did not distort.
The implementation of referencing across models was another animation tool in The Incredibles that broke with past practice. This standardization of a basic common rig structure for all characters made it easier for animators and TDs to share poses and facial expressions between characters. Some characters, however, had multiple rigs. The standard rigs were limited to "normal" squash-and-stretch articulation, but a few special-case rigs were developed for complex distortions. For example, Helen (Elastigirl) in her stretchy suit, or the transformation of baby Jack Jack into a monster. There were also at least two versions of the rig for Bob the Dad: one for the fit superhero version, and another one for his fat version with a gut and shorter legs. Switches inside all models were used to preserve old behaviors by linking different rig versions for each character. A lot of the character articulation work in The Incredibles was done by new employees at Pixar, and the passion for their work and talent are a testament to what the computer animation industry is all about.
Performance Capture and Emotion
From the animation point of view, the initial intent in Robert Zemeckis' The Polar Express was to create computer-generated human characters that were not keyframe-animated cartoons. The scope and the scale of the technology assembled and developed at Sony Imageworks (called ImageMotion) to make this task possible is impressive. The Polar Express' production teams used motion capture technology in an innovative way and developed a unique production pipeline.
Unable to use keyframing as the primary animation technique, motion capture remained the obvious choice for animating the somewhat realistic-looking human characters of The Polar Express. Fortunately, the teams understood what we have learned after years of capturing motion: that there is big a difference between plain motion capture and performance capture. While motion capture seeks to record a cold sequence of moves (ideal for a fast-paced computer game fight or sports competition) performance capture seeks to record the emotion and the intention contained in the way an actor moves and pauses.
The Polar Express uses the later approach, and for that they assembled one of the most complex capture systems ever: four Vicon systems linked together, with 72 cameras in an area measuring 10 feet square. This configuration allowed the realtime body and face capture of up to four actors interacting with one another. The capture of facial performance was done with 152 facial markers, each measuring about two mm in diameter. The data obtained from the facial markers was converted to a muscle system custom-designed for this production, and the facial rigging was driven by the muscle compression for each muscle represented in the system. (For those of you who are curious, The Facial Action Coding System (FACS) developed by Paul Eckman and used in Gollum's facial animation system was not employed in The Polar Express.) But in spite of the impressive performance capture setup it was difficult, sometimes impossible, to capture reliable motion data of eyelids, eyeballs, mouths and fingers.
In addition to their performance capture innovations, The Polar Express production teams also came up with a new production pipeline approach to integrate captured data with cinematography and animation. Large scenes, for example, with captured performances were initially created without a specific camera. This was very different from traditional animation where the scene layout and staging is always storyboarded and laid out from a very specific point of view before the animation stage. This initial scene, called "rough integration," contained only body motion and it could be played back in realtime from any angle by a director of photography (DP). This approach allowed the DP to establish shots by using a "wheel" interface for positioning and moving the camera in the scene while the rough capture was being replayed in realtime, in a mode similar to live action. This innovation represents an interesting twist on the traditional animation pipeline.
The shots continued ahead to "full integration" of body and facial capture after they were approved by the director and the editorial team. Once this stage was finished the shots moved on to the animation department, where the original performances were fine-tuned in different ways. Knowing the capture limitations mentioned earlier one would imagine that only eyes, mouths and fingers were keyframed during the animation stage, but in looking at the finished movie I found minor glitches in these areas and overall facial and body motion. This, in my opinion, points to the fact that some of the captured performances were clearly edited, possibly changed altogether. It is difficult to know exactly how much keyframe enhancement actually took place in The Polar Express, but knowing and understanding these facts will certainly help future users of this performance capture system optimize their work.
When judged from a strictly traditional animation point of view, I think that the style of many of the performances is a bit too straight-ahead for the stylized look of The Polar Express models. A few more keyposes, motion holds and clearer silhouettes might have helped make some of the action read better. Perhaps this is a matter of opinion and stylistic preference, but the lack of consistent emotion in most of the characters' faces is not. While the overall quality of the body performance capture seems consistent and believable, the same cannot be said of the eye animation, in particular.
As mentioned earlier, eyelid, eyeball and mouth animation, all crucial components of facial expression, were keyframed during the animation stage and not captured from the actors' performances. All throughout the movie the motion of the eyelids is minimal, giving some characters a flat look, and too many eyeballs seem focused at infinity. These minor but persistent animation inconsistencies end up distracting, and they represent the weak link in The Polar Express. The odd facial animation style is as if the actors in a live-action movie had a facial twitch every few minutes: no matter how good their performances were, their twitching would surely confuse or dilute some of the emotional intent and take away from the believability of their characters.
Some of the production techniques and pipeline developed for The Polar Express are remarkable, and while the movie was crafted with first class storytelling and rendering, I wish there had been more blending of performance capture with keyframe animation, and I can't help but wonder how this movie would have looked if it had been produced as a live-action movie with human actors and digital visual effects.
Isaac Kerlow is author of The Art of 3-D Computer Animation and Effects and former director of digital production at The Walt Disney Co. He is also a longtime active member of SIGGRAPH and the Visual Effects Society, and currently working on an independent animated feature.