In Beowulf, a semi-photoreal Angelina Jolie with her actual voice is a hard sell.

Some Practical Considerations

Now, try visualizing the Uncanny Valley in 3-D: we have a number of continua forming the geometry. Below is a non-exhaustive list of Uncanny Valley parameters:

Look: Cartoonish/Photoreal (Nestor Sextone; Aki Ross from Final Fantasy)

Morphology: Monster -- Human (Davy Jones); Human (Angelina Jolie as Grendel's mother in Beowulf)

Behavior: Stylized -- Recognizable (Davy Jones; Angelina Jolie)

Face: Unfamiliar/Familiar (Aki Ross; The Conductor [looks like Tom Hanks])

Voice: Character -- Recognizable (Lucille Bliss as Crusader Rabbit; Angelina Jolie)

Animation Style -- Squash-and-Stretch tweaked MoCap (Roadrunner; Davy Jones)

In general, the farther to the left on each of these continua, the easier the character is to sell. The farther to the right, the more human -- like/familiar/recognizable; and difficult the illusion is to sell. Another thing is, of course, that if you put squash- and-stretch on a photoreal virtual human, it doesn't work. However, at least one expert that I spoke to said that if a little squash-and-stretch had been added to the animation in Polar Express, the characters would have been more acceptable, considering the shading choices made. In Beowulf, a semi-photoreal Angelina Jolie with her actual voice is a hard sell.

I've only mentioned six parameters and they impact each other in serious ways. For example, If we did everything right, but had Lucille Bliss, providing the voice for Jolie as Grendel's mother, it would be disaster, so let's simplify and not go there. By backing off on the photoreal and making her slightly less real, she becomes easier to accept. In our minds Angelina becomes a real person playing a cartoon character -- much easier to buy, than a cartoon character playing a real person. Then there's MoCap. It doesn't work well for cartoon characters, just as squash-and-stretch doesn't work for life-like photoreal characters.

The Uncanny Valley is filled with paradoxes, isn't it? For example, we accept Davy Jones as human because he has a fully rendered human personality. It is completely counterintuitive, but in moderation, body morphology going away from human actually helps us to see the character as human. In Davy Jones, we see a human playing a monster, something we're familiar with. In fact, Davy is entirely synthetic with only Bill Nighy's captured data stream providing a shadow of humanity. As you can see, the creature's morphology does not prevent the actor's humanity from shinning through. That's the key: finding and finessing the humanity.

As an aside, the artists and engineers at Electronic Arts discovered some of that when they reverse engineered real-humans to make them appear virtual in a game cinematic. I was told that they had to remove such things like skin pours and arm hair, and that they gelled the real hair to make it seem less real. Habib Zargarpour, vfx pioneer and art director at EA, told me: "We had to remove all the stuff that virtual humans don't have right yet in order to get believable virtual humans -- played by real humans."

Debevec's Lightstage 3 facilitates detailed high dynamic range analysis of how light interacts with the human face. © Paul Debevec.

The Technology We Need

Artistry and psychology are keys to creating believable characters at this stage of evolution. The best technology in the world will not help if we don't know what to capture and how to apply it. A great human performance is the basis for all virtual human work at the present. Sure technology like Softimage's Face Robot are finding reasonably credible ways to simulate human performance from limited data, but full believability remains a ways off. Assuming a great acting performance like Nighy's, we need to know what to capture and what to do with it.

In a recent interview with Variety's Anne Thompson, vfx whiz Rob Legato intimated that the keys to crossing the Uncanny Valley are to avoid realism and to liberate the director from the computer. Legato should know: he has provided a pioneering director-centric system for James Cameron on Avatar, allowing him to incorporate a flexible, live-action-style methodology as part of the next-gen virtual filmmaking process.

The technologies most in need of advancement to achieve what we want also involve subtle performance capture. Capturing eye expressions, including the muscle ridges around the eyes is very important. That includes the Saccadic eye movement, which is the rapid, constant eye movement that changes the focus of the retina as it scans the scene. Often neglected but critically important would be the inside of the mouth. Tongue and teeth/jaw movement is almost as critical as the eyes, and then there's spittle. Then there's all that secondary movement like fat and hair. Hair movement is more critical than you might expect. Believe it or not, you can get the whole thing right and have the hair move just a little bit wrong, and you blow the entire illusion. Final Fantasy was a great example of distracting hair.

Render quality is also essential to believability. I have to mention Professor Paul Debevec's work in studying the complex process of how light interacts with skin and flesh that is helping immensely. Also, Christophe Hery at ILM contributed both his artistic and his technical skills in rendering Davy Jones' skin.

Creating a virtual human actor is a complicated process with no room for sloppy anything. It's a synergistic process among art, technology and psychology. We have to discover all the key human factors and then obsessively apply them and tweak them, until we have it right. Only then will the average audience member accept our illusions as real with comfort and enthusiasm. I'll continue my research in this arena and tell you more as I discover it. Let's hope we all get it right in the near future and move way beyond that nasty deep uncanny gully. The Holy Grail is a fully human looking, perhaps recognizable, virtual human, which we can all believe in without dissonance. I figure two more years with luck.

Writer's Note: I swapped ideas with a number of experts while preparing this article. Josh Kolden of Crack Creative and Christophe Hery at ILM were particularly helpful along Professor Paul Debevec and several people at studios who prefer to remain anonymous. There were also dozens of people over the years that have helped me with this research. Thank you all.

Peter Plantec is a best-selling author, animator and virtual human designer. He wrote The Caligari trueSpace2 Bible, the first 3D animation book specifically written for artists. He lives in the high country near Aspen, Colorado. In addition to his work in vfx and journalism, Peter is also a clinical psychologist with more than a decade of clinical experience. He has spent several years researching the illusion of personality in animated characters. Peter's latest book, Virtual Humans, is a five star selection at Amazon after many reviews.