Image courtesy of Deron Yamada. © 2004 DYA367.

I may have been wrong last month in A Coruña Spain, when I predicted that it would be two years before we establish a beachhead on the far side of the Uncanny Valley. (If you don't know what that is, Google it immediately.) While attending Mundos Digitales, I went to a presentation by Dr. Paul Debevec on his latest work for USC's Institute for Creative Technologies, Computer Vision Lab. He talked about his sophisticated LightStage research into mapping faces in extreme detail. He mentioned that he was working with Image Metrics' Santa Monica, California, office, developing extremely high-resolution reference data for an advanced new process in markerless face capture and animation. He was pretty sure Image Metrics was about to achieve a breakthrough in virtual human face animation. Whenever Debevec gets involved in something, it's going to be done creatively and properly as well. Since I was there speaking about crossing the Uncanny Valley, this news immediately caught my attention.

I immediately Skyped my friend, Patrick Davenport, exec producer at Image Metrics. When I asked what they were doing with the "Emily" project, he hooked me up with Oleg Alexander and David Barton, the development gurus behind the shrouded new process.

What I discovered gives me hope that Image Metrics may really be fast upon a major breakthrough in face performance capture and animation. The Botox Syndrome, which infects virtually all face capture systems to date, may soon become a thing of the past.

Emily O'Brian being digitally cloned in USC's Light Stage 6 at the Institute for Creative Technology. All images courtesy of Image Metrics and USC's Institute of Creative Technologies.

The Company
Stepping back a bit, Image Metrics does face capture without using markers glued to the face. This makes life more comfortable, and certainly less complicated, for people having their faces captured. It sounds like a dreadful thing and it can be. Image Metrics' approach to facial animation evolved from research in the field of computer vision. Three Ph.D. candidates in computer vision at the University of Manchester: Kevin Walker, Gareth Edwards and Alan Brett, developed technologies for tracking faces and analyzing medical images as part of their dissertation projects. That technology has to intelligently identify faces in more than one position and orientation, for example, for use in security applications. This requires analysis capable of producing the kind of data needed to track facial behavior. In fact, they can track virtually on a pixel-by-pixel basis with no face markers. They were soon able to extract reliable and accurate facial data from ordinary 2D video. They do it by keeping frame-by-frame track of tiny face irregularities, even pores, as well as head orientation and rotation. The concept opens up a lot of interesting possibilities like tracking classic performances in old movies and bringing old actors back to life… virtual life. They actually did that with Richard Burton.

Together the three friends formed Image Metrics back in 2000 in Manchester, U.K., to exploit the promise of their new technology. They've had great success, and have produced some remarkably accurate face animation; but to date, not accurate enough to completely fool most of us.

To make face behavior believable in virtual humans takes an enormous amount of synchrony between art and technology. So far, technology alone has not been able to do it. The Botox Syndrome is mostly the result of inaccurate movement of skin across the face, and eye movement that is just wrong. I confess it drives me crazy when I have to watch it.

It also drives the folks at Image Metrics crazy as well, which is why they've worked so hard to develop their facial animation technologies. In addition to tracking the areas that marker-based systems track, Image Metrics has found a way to accurately track places markers just can't reach, such as the eyes, lips, tongue and teeth -- the areas that display the subtle emotions that make us who we are.

In order to really prove that their technology could accurately portray reality, Image Metrics Lead Rigger, Alexander and Head of Production, Barton, immersed themselves in "Emily."