Karl Cohen sits down with Rob Coleman, Episode II's animation director, and learns how the ILM team combines many different techniques into one seamless digital world.
Combining digital characters and real actors in the same shot truly put the ILM team's digital artistry to the test. © Lucasfilm Ltd. & ™. All rights reserved. Digital work by Industrial Light & Magic.
The latest Star Wars feature is a remarkable technical tour-de-force. Almost every shot in the film contains some digitally created element, whether it be a background, a vehicle, a prop or a foreground character. Of the approximately 2,000 shots in the film, 928 of them contain characters created by computers. That comes to about 69 minutes of screen time.
It took a team of 60 animators and another 340 artists and technicians at George Lucas' Industrial Light & Magic (ILM), San Rafael, California, to create all the post-production elements in this film. Besides animators, the crew included technical directors, modelers, view painters, compositors, miniature artists and lots of other specialists.
When AWN was invited to interview Rob Coleman, the animation director of Star Wars Episode II: Attack of the Clones, we wanted to learn about his role in the production, along with how a movie of this scale is broken down so that it can be made. Who decides what techniques will be used to achieve the effects for which the script calls? Where did they use models instead of computer-generated backgrounds? What was animated using Macintosh computers? Where did they use motion-capture?
Animation director Rob Coleman describes the first task on Episode II was to define the technical "gotchas." Photo credit: David Owen.
Coleman says George Lucas begins to make major decisions almost from the start in the pre-script phase of the production. Lucas tends to begin his Star Wars projects by giving the art department and production designers at Skywalker Ranch (a magnificent facility in a hidden valley in rural Marin County, California) an idea of the worlds he wants to visit in the movie. "In this movie, we have a world completely covered with water and we have another world that we called 'the rock world' until we were given the name of it. In each of these worlds, we meet creatures that have not been seen before in the Star Wars universe. I'm not in the early meetings where George describes verbally what his vision is, where his imagination is leading him. These meetings are with his production designer Doug Chiang, the design director, and other designers and illustrators working with him." They then create hundreds or thousands of drawings for each idea discussed. It might take one month or several before, "George decides, yes, that is what that character will look like, that's the costume they will wear, that's the building they are going to stand in and that's the vehicle they are going to fly in."
"At that point, I'm invited out to Skywalker Ranch with other key creative people from ILM, to have a presentation of the art work. The first meetings of Episode II involved John Knoll, one of the visual effects supervisors, and myself. We had worked on Episode I: The Phantom Menace together. We attended a series of small, hour long, meetings where they presented some of the characters, some of the creatures and some of the environments. We saw rough pencil drawings and beautiful production paintings. They also showed us small little maquettes (statues) of the characters."
"What we were interested in at that point was what were going to be the technical 'gotchas.' They are the things that an artist can do with a flick of the wrist, that will take a team of five computer scientists six months to try and figure out how we are going to do it. Gotchas are things that are technically impossible to do. A good example from Episode I is when the script said, 'The Gungan army marches out to meet the Droid army.' One sentence. It took us 10 months to figure out how to do it. They were innocent lines in the script and they were certainly innocent drawings on the wall. You might say, 'Isn't that interesting?' until you try to figure out how we are going to get that to work."
The Number One Challenge -- Elaborate Costumes
"One of the first things that we noticed when we first met with George was the very elaborate costumes that these digital characters were going to be wearing. We were happy with where we had gotten with digital clothing on the first movie, but we realized immediately that clothing would be at the top of our technical challenges list for Episode II."
"In the Star Wars world, we have digital characters that are composited beside real people. In a film like Shrek, everything is animated. In Star Wars, we have digital characters living in the 'real world.' The audience can compare Natalie Portman's beautiful costume to Yoda's robe in the same shot, but Yoda is digital and his clothing is not real. It's virtual. The audience has to see them as equal. It has to have the right weight, the right billow, the right follow through. The audience has to believe that these two characters are existing in the same space at the same time. If they don't, we have failed as special effects artists."
"It's a very high bar to get over. We walk around all the time observing people wearing clothes so we have a very rich understanding of silk, burlap, leather and how other fabrics look when they are moving. When we apply that to digital characters we have to be amazingly accurate. That was our number one technical problem."
The photo-realistic backdrops in the Clone War were all created digitally. © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic.
Challenge Number Two - Digital Environments
"The next thing we focused on when we saw the beautiful production paintings of these worlds was: how elaborate were the cityscapes and the scenery going to be in this movie? At that point, we were informed at how little was actually going to be built on the stages in Sydney. John [Knoll] and his group of people began to work on how we could improve our digital environments. John and his team worked for months trying to perfect that."
"One sequence ILM is proud of is a chase through a cityscape early in the film. At the other end of the spectrum, Ben Snow, another one of the visual effects supervisors, worked on the Clone War. He created photo-realistic rocky environments including canyons, buttes and plains."
Creating realistic movements and reactions are key to making digital action sequences like this one from the Clone War credible. © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic.
Challenge Number Three - Rigid Body Simulations
Rigid body simulations is the creation of believable weight and gravity on objects that are falling and exploding. Coleman says, "Imagine that you have a clone who is running along and gets hit by a concussion wave. He has to fall and 'clatter' on the ground. That's very difficult for an animator to animate, to get the right bounce and weight without spending a great deal of time on it. The team that worked on the clothing simulations came up with solutions so we could have vehicles exploding and pieces flying off them or droids hitting the ground and clattering. We used our new software to create realistic reactions every time two digital objects hit each other. It's amazing how many times that shows up. George knows his audience."
Animating Yoda
Coleman says, "The biggest technical challenge for me was animating Yoda. That was due to Frank Oz having created that character in The Empire Strikes Back as a puppet that we all know and love. This time, Yoda, who is supposed to be 2'll" tall and 874 years old, is much more active and prominent. He actually runs and has a light saber battle. So he had to be a fully animated character in this film, not a puppet. That decision was made early on because George indicated Yoda was going to have to move quickly. In his earlier screen appearances, he moved slowly with a cane. I wanted to make sure this character was reminiscent of our recollections of what he was like in 1980 and 1983."
CG model supervisor Geoff Campbell (left) and animation director Rob Coleman discuss how to preserve the charm of the puppet Yoda from The Empire Strikes Back while animating him digitally for Episode II. Photo credit: David Owen.
"I studied Frank Oz's work from those classic movies with a great team of animators. Geoff Campbell, the modeling supervisor on this film, built a beautiful digital model of Yoda. We took a great deal of care to animate him carefully and lovingly so he remained true to his character. I spent a lot of my time supervising his performances in the movie because he's in a lot of it."
When the feature was being shot in Sydney, Australia, Lucas had Coleman present so he could be consulted about how the post-production images would work with what was being filmed. Lucas would ask him from time to time what Yoda's movements would be in a scene so he could direct his actors properly. Coleman says, "We had a lot of discussions about how slowly Yoda would be speaking. We didn't have the luxury of having Frank Oz there with us on the set when we were shooting the live-action. [Oz provided Yoda's voice in this and earlier films.] We had one of the actors perform his lines off-camera so we could get the right cadence in the scene. That worked out really well."
An all digital Yoda for the new century. © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic.
"We had a stand-in rubber Yoda puppet from a previous movie so we could see what the lighting was going to look like. We photographed that puppet sitting or standing in the scene with the lights on it so the technical directors and the compositors would have a really great reference frame when it came time to light the digital Yoda."
Animation Done On Macs
Despite the price differences between high-end Silicon Graphics workstations and Macintosh products, Coleman says, "When you are dealing with top level artists and technicians, they can pretty much create the same look and feel on any platform. Some platforms have better modeling packages, some have better renderers, and some have better atmosphere. We have a good library of in-house software that we have written and off-the-shelf software that we have bought, so that we can create the imagery that we need."
The flying R2-D2 was created using Macintosh computers. Even George Lucas thought it was a prop! © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic.
There is "a tiny, tiny amount" of Mac-based animation in the film. Its use was based on the availability of Billy Brooks, who animates on a Macintosh system. Coleman says, "The computer graphics department here is enormous. At any time we can be working on from 5 to 10 different films. Star Wars was always the biggest project, but it was never the only one. Some times we are fully booked. We do have some talented artists like Billy Brooks who worked on the R2-D2 shots using Macintosh. It looks the same."
Not Everything Was Done With Computers
"When we are pre-planing, we are looking at expense, but we are also looking for a balanced crew. It doesn't service us very well to give all the work to the computer graphics department. We don't have the manpower to handle it, and we have very talented people over in the model shop." He says that Episodes I and II have more miniature models in them than there were in the original Star Wars features.
Physical models (real, not virtual) "were used in the Queen's city on the planet Naboo in Episode I. On this film, there was some model work used in the city of Coruscant, in the droid factory, and in the sections of the big arena battle supervised by John Knoll. There was an enormous arena built that might have been 14 feet high and possibly 65 feet wide. It was huge. We even joked, 'When is a miniature not a miniature?' That miniature was representing an arena that held 100,000 creatures. Then, we put CGI characters in those stands."
Another miniature set was created at the last moment for "part of the battle that we call the Dune Sea, a sand dune sea. That was a miniature set shot on one of the huge stages here at ILM. That section was added so late in the film that the decision was made to create that as a real miniature because there wasn't time to create a digital version of it."
ILM does a lot of work where they combine several techniques in a single shot. They may have a matte painting, miniatures in the background and can add digital actors along with real actors shot against blue screen. The era of glass mattes is ancient history as the computer can do the same work.
"Some people in the digital matte department have been with us for 15 or 20 years. Paul Huston has been with us 25 years. He made the transition from traditional matte painter to digital matte painter many years ago. They still use some of the old techniques. For example, they still shoot miniatures as elements that are going to go into the matte painting and then blend them in. Another thing Paul and some of the other artists do now is what we call 3D matte paintings. They are matte paintings that allow you to do compound camera moves in them. There is some 3D geometry in them that allows you to have parallax on a building. Now you can pan around a building a little bit or dolly into a shot better. It is a modern version of what Disney created with his multi-plane camera stand."
Is C3PO a digital image or a hard surface model here? With ILM's mastery in both CGI, puppet and model making, it's hard to tell. © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic.
Mechanical Puppets Or Animatronics
In Episodes I and II, there are people wearing masks that have radio controlled puppet operators running them. In the new film, there is "a bad guy who wears a rubber head that has radio controlled eyes and mouth articulation while the actor walks around and can move his arms." These devices were not operated by ILM crews. Nick Dugman operated this type of system in Episode I and Simon Williams from England, controlled them in this film. They were shot in Sydney.
R2D2 and C3PO are what ILM calls hard surface models. The models have radio controlled features as well as actors inside the suits. In Episode II, there were a couple of scenes where it was either impossible for R2D2 to be operated by radio control and/or for Kenny Baker to move safely inside the model. Coleman says, "There are 2 scenes I can think of right off right away that were impossible or too dangerous to do. One was a scene of R2 walking up stairs. The prop could never do that. The other one is R2 flying. It made sense to do a digital version of him as he was going to be flying through a digital environment. It is totally photo-realistic. You'd never know it was a digital model. We even fooled George. He thought we photographed it!"
Similarly, there are some scenes where C3PO had to do dangerous stunts. "We had two concerns. One was hurting Anthony Daniels, the man inside, and the other was hurting his suit. He might end up breaking his skull if he fell over. That is always a concern as Anthony can hardly see out of that head. There are always people around to catch him if he ever tripped. There are a couple of scenes where C3PO is in some pretty elaborate stunt situations that were really dangerous, so it made sense to create a digital version."
ILM refers to both of those characters as hard surface models as "they don't have any flexible clothing. C3PO has some soft material between the joints, but that's pretty limited. In both cases, we have beautiful reference frames as you can photograph the suit from every angle under ideal lighting. We give the information to what we call our 'view painters,' the people who create the texture maps." Coleman asserts that one can't tell the difference between the computer generated image and the footage of the model: "You just cannot tell."
A Motion-Capture Finale With A Cast Of Thousands
Most of the clones in the film are moved using motion-capture. They had to be moved in a realistic way as they are supposed to be human clones inside of armor. There are shots of clones marching, walking, standing and shooting. In some shots, there are thousands of clones fighting thousands of droids. It would have been too time consuming to try to animate those sequences by hand.
Motion picture capture supervisor Jeff Light directs "droid" James Tooley. Photo credit: David Owen.
With so many elements, a procedural program where the models were automatically moved in battle would surely have been a blessing. However, for Coleman there was no such short cut. "Unfortunately, we haven't reached that level yet. We did a lot of motion-capture sessions where we had people running and shooting, running and falling, jumping and shooting, and marching. We created a library of elements with all these different actions, and then a choreographing team choreographed the fights."
"I think of the big complicated shots as having three levels. We have the foreground action that most of the audience is going to see. We have the mid-ground level where you may perceive some of the action. The background action is really just there for texture. You can't see what's going on there unless you still frame it on a DVD." In battle scenes, there may be upwards of six droids or six clones in the foreground. The mid-ground may have something like 20 that are pulled from the library of elements. When they build a mid-ground scene they may have 15 or 20 stock sequences to work with. In the background are what ILM calls "sprites." They are tiny bits of action applied to low resolution clones. You will never be able to tell the difference because they are not big enough to see. So the foreground is mostly hand animation, the mid-ground procedural animation and the background is simply pulled from key frames and motion-capture action. Nobody has to retouch the artwork. If feet are sliding on the ground or arms are passing through somebody's torso, nobody will know. They are just there for background motion.
Jeff Light, James Tooley and Michael Sanders (left to right) on the motion picture capture stage. Photo credit: David Owen.
Coleman says, "We are thinking about the same kinds of things that everyone has been thinking about since people began making movies. Where do we want the audience looking? In the layout of the shot, what is the flow of the action? We have a very educated idea of where people will be looking because we are forcing them to look there. We make that section of the shot look good and then we layer in everything else. We storyboard the action just as you would on any animated film. We create animatics, so we prove to ourselves in a rough cut they're all working. And then, we layer in the final animation. We are not surprised when we see the final results."
Summing It Up
Coleman summed up his involvement with the film by saying that pre-production was getting an idea of what is coming, production is being there to witness how the original ideas get changed and post-production is seeing how it all fits together and how it changes once again. He is used to changes. Some happen when they are shooting. George Lucas has been known to restage a scene so one that was planned as a series of close-ups ends up being long and medium shots. That means more characters, additional set details and other elements may be needed to complete the scene. Coleman is used to seeing the environment change from the pre-production paintings to what George wants when they are filming. "It might be more characters, more windows, more architecture. In terms of characters, costumes can change from the drawing board to the final execution because George may make an aesthetic decision to change the look of a digital character."
Yoda and Obi-Wan Kenobi (Ewan McGregor) with Jedi children on Coruscant in this shot that successfully combines digital work with live actors. © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic. 
Rob Coleman is most proud of how the digital image of Yoda captures the personality of the original puppet character. © Lucasfilm Ltd. & TM. All rights reserved. Digital work by Industrial Light & Magic.
Post-production has to deal with what was actually recorded and Coleman reminds us that everything was shot with digital cameras -- no film was used in the making of Episode II. The directors' plans for a scene may have to change if an actor isn't looking where he or she was supposed to look. In post, changes occur when they drop in the digital actors and Lucas realizes the scene will play better if... Coleman says, "Shots can become more complicated or change completely based on the flow of the movie, editing decisions or how the story is being told."
While we didn't talk about Episode III, one brief comment suggests what we might see in the future. Coleman talked about Lucas adding additional scenes and animation after Episode II's live-action photography was supposedly completed. Lucas is accustomed to going back to the Skywalker art department and having them "design new characters, new vehicles, new costumes that we didn't think we needed, but now we do. Or George will write a new scene and it will be shot a year after we shot first unit photography. That happened. There was a whole room that was added about a month and a half ago. [Coleman was interviewed two weeks before the film opened.] A new scene was written and the actor was shot again. We had to quickly create a digital background and I had to put digital characters in with new lines of dialog. George said, 'If I had known you could turn a shot out so quickly I would have added more stuff.' Having all these great toys and great talented artists here, George certainly has fun working with us." I think we can expect even more of the impossible in the next film.
The biggest accomplishment of which Coleman is most proud? He replies without hesitation, "I'm most proud of what our team was able to do with Yoda. I think he's beautifully animated and I think we were able to remain faithful to what Frank Oz had established."
Karl Cohen is President of ASIFA-San Francisco. His first book, Forbidden Animation: Censored Cartoons and Blacklisted Animators, is published by McFarland Publishers. He also teaches animation history at San Francisco State University.
