'Inspired 3D': Lip-Sync and Facial Animation — Part 1

This is the next in a number of adaptations from the new Inspired series published by Premier Press. Comprised of four titles and edited by Kyle Clark and Michael Ford, these books are designed to provide animators and curious moviegoers with tips and tricks from Hollywood veterans.

Since animation is a communicative language, the more you expand your understanding of how motion and pose can be applied, you will expand your vocabulary and find the necessary elements to add depth and subtlety of meaning to your communication. Facial animation is a particularly challenging aspect of character animation, primarily because it has such a demanding need for both subtlety and broadness in what is being said.

Facial Animation

For the longest time, I relied on my innate sense of timing and expression to animate faces, relying on the gut instinct that Id been blessed with. Unfortunately, that only carried me so far before I started falling back on old habits and patterns. I then got down to the nitty-gritty and analyzed how to approach facial animation in a moderately systematic fashion. My goal was to open up a world of principles to be used, mixed and matched to conjure up new combinations and possibilities. Keeping the fluidity of all great facial animation at heart, I want to show you the techniques, principles and thought processes you can experience while animating faces.

[Figures 1 & 2] The characters smiling mouth and sad eyes (left) are contradictory. The characters sad mouth and sad eyes (right) express the same thing.

The face, perhaps more than any other part of the body, is a window into the soul of a character. As such, there is a thin line between good and bad facial animation. The margin of error in executing facial animation is slim, more so than other parts of the body. Having said that, theres plenty of room for loose interpretation in facial animation. Lip-sync and emotional posing of facial features can be as broad or as subtle as the detail of the character or the need of the moment allows. This opens doors to portray thought, motivation and emotion in your character in rich, subtle, and powerful ways. To achieve these results, youll have to think about facial animation in a way that is slightly different from other parts of character animation.

Singularity of Message

Animation is, at its core, a communicative language. As such, your primary goal is to clearly communicate your given message in such a way that there is no ambiguity or doubt about your characters thoughts and actions. When youre trying to convey a message, you must provide clarity. If your message lacks clarity, it becomes muddled, confused and oftentimes unintelligible. Say only one thing at a time and make sure your whole character is saying that one thing. Your message can be simple (Im happy) or complex (Im happy, but not quite as happy as I thought Id be), but it must be singular. So make sure your whole characters face is saying the same thing at the same time. (See Figures 1 and 2.)

[Figures 3 & 4] The sad pose of the body matches the sad look on his face (left). The happy face is contradicting the body (right).

Build on the Body

Facial animation cannot save poor body animation. You cannot hope to rescue badly timed, poorly motivated or sloppily executed body animation with great facial animation. Unless the shot is a close-up, facial animation is more often a flavor enhancer.

Body language accounts for 90% of the emotional communicative weight of a character. Body language can be read from afar, without a good look at a persons face. Follow the lead of the body language and make your facial animation fit the intensity, energy, tone and tenor of the body animation (see Figure 3).

Strive for consistency in your message. The last thing you want to do is put Im happy! facial animation on a body thats saying I should have stayed in bed this morning (see Figure 4). That is unless the situation calls for a lie to be told. All things in facial animation are relative and subject to the needs of the shot, which is why there are so few rules to go by.

[Figures 5 & 6] A symmetrical face (left). An asymmetrical gesture (right).

Asymmetry

The initial temptation in all animation is to be too symmetrical (see Figure 5). Asymmetrically animating the face creates endless possibilities for facial expressions. Peoples faces are bland and boring when theyre symmetrical. However, raise a single eyebrow and now you have a sarcastic look, or perhaps a quizzical, cynical or mildly surprised look.

By combining various asymmetries in your face posing, you can mix and match to capture just the right expressions to clearly communicate an idea. Rotate the jaw left and right to add flavor. Raise one eyebrow higher than the other to add punch to a facial expression. Sneer one side of the lips to break up the flat even line of the mouth. Cock a half smile on a character to hint at an underlying motive. Leave the jaw slightly more slack on one side to impress the notion of utter shock. Play around with the combos of poses and asymmetry to unlock great little gems in facial animation (see Figure 6).

[Figures 7-9] The character is smiling with an asymmetrical pose (left) setting up the move to an angry face. The face with the cheeks and nose (center) NOT being used. A face with the connectors tying the whole face together (right). The cheeks and nose are receiving the proper attention.

The most dramatic emotional shifts can be given higher intensity by shifting the asymmetry in reverse. For example, pretend the character is smiling out of the right side of his mouth and his right eyebrow is down. (See Figure 7.) Now he becomes angry and shifts to a frown on the left side of his mouth and raises the right eyebrow higher than the left. This reversal gives the characters internal emotional shift an extra kick, helping it to read more clearly.

Facial Connectors

A telltale sign of inexperienced 3D facial animation is shown when the characters mouth and eyebrows are moving, but the vast dead sea of face in between never moves. The face lacks a holistic connection within itself. Often, this disconnect between the eyes and the mouth will result in a confused message.

The trick is to think of animating the entire face as a whole, not just animating parts of the face. Because of the complex musculature of a face, it is nearly impossible to move the jaw without the muscles and skin all around it being affected, even all the way up to the eyes (see Figure 8). If you feel like a good lip-sync animator and a good eye- emotion animator, but youre looking to put your work over the top, then youre looking for connection. The primary connectors in a face are the cheeks, the nose and the ears (see Figure 9).

From a technical standpoint, you should try to build as much forethought into your facial morph targets as possible. Try to get the facial connector areas incorporated into your target building so that youre maintaining connectivity for your face. Technically, from a morph target point of view, youre going to want to make sure that you build push and pull into the nose, cheeks, and ears of a character. These face parts are highly driven by the underlying muscles used to make facial expressions. Here are a few tips to keep in mind when building your facial shapes:

• For the mouth corners up, make sure the ears are pulled up a bit.
• For the jaw open, make sure the ears are lowered a tiny bit.
• For the sneer, wrinkle up the nose.
• For the smile, make sure the cheeks rise up.
• For the jaw open, elongate the cheeks and slightly stretch down the base of the nose.
• Dont forget about the neck muscles used to move the jaw and mouth corners.

Probably the single most useful technique to getting the face to seem connected and whole is to key-frame the entire face at once. The exception is the mechanical operation of lip- sync; that is usually treated as a separate issue and will be discussed at length shortly. But for emotional and expressive posing, you can often work on top of the underlying lip- sync. A person can say Oh, yeah, Im doing fine and really mean it, or he can be snipping back sarcastically. In both instances, the lip- sync execution doesnt differ much, but the entire face posing and expression is vastly different.

Its in this realm of emotional expression that I am suggesting that you pose and animate the entire face as one unit. Treat faces like body poses. Faces have distinct and clear poses just as much as the body does. Ignoring this rule may result in a face that is haphazard in regard to timing and impact. You can certainly offset the key-frames in your finessing stage, but from early on you should use broad strokes in blocking in your characters basic shifts in facial expression.

If the character is happy, key the whole face to be happy. Then, if the character shifts to sad set the whole face to be sad. (See Figure 10.) If youre doing lip-sync on top of this shift, treat the lip-sync as a distinct technique. Animate the lip-sync first; then go in behind it and modify the face to fit the emotion.

A persons facial shift will tend to be a moment in time. There may be offsetting, especially between the eyes and the lower face, but the viewer should get the strong impression that the shift in facial expression is a very distinct and identifiable moment. The various shapes that make up an emotion will change with similar timing.

When animating the face, dont try to do too much at once. Lip-sync needs to have flow from one sound to the next, with relatively few holds. Eyebrows and overall expressions should be singular tending to be held until there is a solid emotional and cognitive reason to change. Theres a lot of power in skillfully changing from one clear emotion to another in a shot.

The 12 Principles Applied to Faces

One of the temptations in animation is to insert the 12 principles of animation as defined in the watershed tome, Disneys The Illusion of Life. These 12 principles have become a foundation for understanding animation. However, with the passage of time, they have taken on a sacred aura. As a result, many beginning animators feel they absolutely must insert as many aspects of the Magic 12 as they can into each and every shot. However, merely adding overlap for overlaps sake is misguided.

It is accurate to say that all aspects of the Magic12 have their proper place in facial animation. You do want to keep the face fluid, alive, and organic, so adding offsets and follow-through on the occasional emotion is needed. However, if you apply overlap and offset too much, the face becomes rubbery and loses its meaning. The face, more than any other part of the character, is a billboard of intent. Everything that the face does must be motivated by something. Any extraneous motion in the face will generate confusion. To do more merely for the sake of the principle of it is moving things without understanding why. And with facial animation, WHY is king. So as with all animation, know what youre going to apply and why; then go ahead and apply it.

To learn more about posing and staging, character animation, walks, tools of the trade and other topics of interest to animators, check out Inspired 3D Character Animation by Kyle Clark; series edited by Kyle Clark and Michael Ford: Premier Press, 2002. 268 pages with illustrations. ISBN 1-931841-48-9 ($59.99) Read more about all four titles in the Inspired series and check back to VFXWorld frequently to read new excerpts.

Keith Lango is the computer graphics supervisor of the feature film at Big Idea Productions Inc. in Chicago, makers of the top-selling childrens video property Veggietales and 3-2-1 Penguins! Keith got his start in CG in the early 90s and has held positions as an illustrator, a senior animator, an animation supervisor, an assistant director, a CG supervisor and a writer. Keith has also co-authored and co-illustrated a childrens book as well as personally developed several award-winning short animated films. He lives happily with Kim (his wife of 14 years) and his three children: Candice, Laura and John Mark.

Author and series editor Kyle Clark (left) and series editor Mike Ford (right).

Series editor Kyle Clark is a lead animator at Microsofts Digital Anvil Studios and co-founder of Animation Foundation. He majored in film, video and computer animation at USC and has since worked on a number of feature, commercial and game projects. He has also taught at various schools, including San Francisco Academy of Art College, San Francisco State University, UCLA School of Design and Texas A&M University.

Series editor and author Michael Ford is a senior technical animator at Sony Pictures Imageworks and co-founder of Animation Foundation. A graduate of UCLAs School of Design, he has since worked on numerous feature and commercial projects at ILM, Centropolis FX and Digital Magic. He has lectured at the UCLA School of Design, USC, DeAnza College and San Francisco Academy of Art College.