Autodesk Reveals the Secrets of the Magic Mirror

files/pictures/picture-91.jpgDreamWorks' big green ogre has been residing on Broadway since last fall, when SHREK THE MUSICAL debuted. Painting actors green or putting them in donkey suits is one thing, but the original animated film had more than its share of moments that might be challenging to recreate in front of a live audience.

Take for example the Magic Mirror that advises Lord Farquaad. In the old days the producers might have settled for an actor on the other side of a pretend mirror, or perhaps a pre-produced piece of animation. Today however, the same technology that brings these moments to life onscreen can perform the identical magic onstage -- and thanks to modern motion capture technology, go one better by doing it live, every night in real time.

The show's producers turned to Autodesk and the company's MotionBuilder software to do the job, and a handful of media reporters were invited to a Friday night performance to see MotionBuilder in action. As they sat in the audience during the show's set-up, lights were hoisted into the rafters, scenery backdrops descended into view and rose again and the sound technician could be heard chanting "Donkey-Donkey-Donkey, Fiona-Fiona-Fiona" as he tested each character's microphone lines. Heath Schwartz, Shrek's press agent explained that in the musical's Seattle tryouts the Mirror's performance was pre-taped. "We did one session with the actor to work out the timing and leave holes for the stage actor, then it was all computerized. Then we said we're coming to Broadway, 'it's got to be live, everything about the show's got to be live."

Actor John Tartaglia performs as both the Mirror and the show's Pinocchio. Schwartz outlined Tartaglia's career as puppeteer on SESAME STREET and in the puppet-populated Broadway musical AVENUE Q. (Tartaglia also brings the giant-sized Dragon head that appears midway through SHREK to life.) "After holding a puppet in his hands for many years, John's now playing a puppet."

Every night (and twice a day on Saturdays and Sundays) Tartaglia removes his Pinocchio makeup mid-show and dashes up 5 flights of backstage stairs to a tiny motion-capture studio. Paul Deavila, Shrek's 'moving light and live animation technician' explained that Tartaglia arrives only about a minute before he goes live as the Magic Mirror. In that time 20 reflective motion capture markers have to be placed on his face "within a millimeter of where they should be." Once 'made-up,' Tartaglia sits in front of a dozen Vicon motion capture cameras and begins his performance as Farquaad's Magic Mirror.

The information from the cameras is captured on Vicon's Blade software and sent via a gigabit Ethernet to a processor that prepares the data for Motion Builder, which performs the 3D modeling. The output is sent to Hippo servers that crop the image to fit on the 20-foot tall, LED-array 'Magic Mirror.'

Tartaglia repeats the entire process in the show's second act for the Mirror's second appearance. (His reprise takes place just after Farquaad uses the mirror to play his own version of Space Invaders with the show's characters taking the place of the advancing aliens.)

The Mirror face the audience sees is identical to the one in the original animated Shrek, because Motion Builder is working from character design files supplied to the show directly from DreamWorks. The entire process takes place, Deavila boasts, not 'basically' in real time, but in real time, period.

A taped backup for the Mirror exists in case of an unexpected malfunction, which according to Deavila has only happened once in over 180 performances. He added that "three different actors can do the Mirror and we have different calibrations for each one" to translate that actor's facial motions into the Mirror's expressions. "It would take a couple of hours to calibrate the system for a new performer."

After the show Stig Gruman, Autodesk's VP of Digital Entertainment gave the company's perspective on its association with the musical. "Autodesk is about the democratization of technology. This MotionBuilder application is very much in that spirit.

"It's a wonderful complement to our 3D software like Maya and 3ds max," as well as Softimage which the company acquired in September. "The exciting part now is figuring out how to capitalize on all this technology in our portfolio to deliver more value to our customers."

-- By AWN Contributor Joe Strike