Summer Previs to the Rescue

[1]

Iron Man [1] used Pixel Liberation Front's entire visualization toolset from pre-production to post for 19 months with a total team size of 25 individuals. All Iron Man images © 2008 MVLFFLLC. ™ & © 2008 Marvel Ent. All rights reserved.

If it's summer, it must be movie tentpole time, which is when VFX is at its busiest and arguably its best -- and previs too. Not surprisingly, Iron Man [2], Speed Racer [3] and Indiana Jones and the Kingdom of the Crystal Skull required the fullest capabilities that previs has to offer as an evolving, indispensible tool.

"Iron Man utilized our entire visualization toolset from pre-production to post," remarks Kent Seki, visualization/Heads-Up Display supervisor from Pixel Liberation Front. "We were on the film for 19 months and our total team size comprised of 25 individuals. The largest our team was at any one given time was 11 artists. I was the only individual artist from PLF to be on the show for the entire duration. We rotated artists and coordinators in and out as the needs dictated…

"We helped the production improve the film by giving the filmmakers the fastest way to conceptualize story/shot/sequence ideas and see the results on screen. They were then able to accurately evaluate the success of these ideas in a cost effective and timely manner. We became the digital 'scratch paper' of the film. This encouraged the filmmakers to repurpose footage, ideas and to continually try and improve sequences. This relentless pursuit was reflected by all the parties involved, especially in post-production. As a result of this experience, we gained valuable insight into motion capture integration and character rigging. In the middle of the show, we purchased our own in-house Vicon motion capture system and began seamlessly integrating it into the visualization. We pushed our abilities in terms of sheer volume of shots and work. We had a tremendous amount of diverse work at any given time. By the end of the show, we had managed to provide previsualization, technical planning, on-set consulting, postvisualization, HUD effects supervision [for controlling the Iron Man armor] and final shots that included graphic design and compositing. More important, we were allowed invaluable access into the filmmaking process through the generosity of Director Jon Favreau, VFX Supervisor John Nelson, VFX Producer Victoria Alonso and Editor Dan Lebental. Their encouragement and attitude created an extremely hospitable work environment. At the end of the day, this film reinforces PLF’s ideas about successful creative collaboration and the potential that this collaboration represents."

TD Brad Friedman took on the task of interfacing with Iron Man suit designer Phil Saunders. Since no final vfx vendors had been hired yet, Friedman began R&Ding suit motion for the eventual digital version of Iron Man. Both Saunders and Nelson were concerned with the physical properties of three-dimensional moving armor, according to Seki. Interpenetration and range of motion topped their list. PLF artist Mahito Mizobuchi worked to model Saunders’ designs while Friedman rigged and tested the suit. The team focused on the abdomen and the shoulder/torso/arm areas. Friedman repeatedly rigged and tested a fully articulated version (metal plate for metal plate) until he arrived at a solution that satisfied the need for appearance of realism while retaining necessary range of motion. The vfx production staff sent these early tests to the perspective vendors as reference. The production awarded the final build and refinement of the practical suit to Stan Winston. Their exemplary work also became the basis for the final digital suits used by ILM, the Orphanage and the Embassy.

[4]

The challenge of creating the digital Iron Man suit [4] included interpenetration and range of motion. The solution had to satisfy the need for appearance of realism while retaining necessary range of motion.

Favreau was also intent that the flying “feel real.” For this task, Nelson motion captured stunt flyers in a wind tunnel to get the proper nuanced motion that Iron Man’s flying style would require. Some of PLF's early tests consisted of putting the MoCap onto their previs version of Iron Man. The results were very encouraging, Seki suggests. This Mocap provided the basic reference from which ILM keyframed their flying animation. The use of Mocap underlined the need to develop a system to incorporate motion capture, but still be able to add keyframe before or after it. Friedman developed a character marionette system that allowed any number of rigs to be used for any one character. The user could then take the character and blend the geometry of a fully weighted character onto any of the rigs (MoCap or keyframe) in the scene at any given time. PLF chose to use this system as well as reference models in Maya since they knew that the character rigs and models would be modified as the job progressed and they acquired new information. "This ability provided a solid base for us to be flexible with vendors and the needs of production," Seki adds. "We could easily swap out the model we made for the Stan Winston suit model, then the ILM one without losing our animation. In addition, as the need arose, Friedman developed the 'advanced MoCap rig' that allowed our artist to keyframe directly on top of the MoCap data to modify it 'on the fly.'

"I initially led the previs effort on First Flight with the Mark II while PLF Lead Kyle Robinson took the reins on the Dogfight. We then split the team between the Escape and Gulmira. Mary Manning and Robinson tag teamed the lead for the Escape while I focused on Gulmira. For these sequences, we tried to get as much staging information from Stunt Coordinator Tommy Harper as we could. In addition, we were allotted valuable time with Cinematographer Matthew Libatique [Inside Man, Requiem for a Dream]. We would try and gleam anything we could regarding his style, lensing and shot selection. His films were 'required viewing' for our artists.

[5]

On Iron Man [5], Favreau used previs as a medium to experiment and a way to get ideas evaluated. For him, previs was a “template,” but not the movie itself.

Seki suggests that Favreau’s approach to previs was a bit unconventional. He used it as a medium to experiment and get as many ideas out and evaluated as possible. He was adamant that the previs was a “template,” but not the movie itself. ILM Animation Supervisor Hal Hickel once speculated, according to Seki, that Favreau’s acting background and his penchant for improvisation may have been the source of this process.

"The notion of continually revising and improving sequences can be a bit daunting; however, if the environment in which you work is supportive and collaborative, as it was with Iron Man, you find yourself embracing the process. It can be infectious. John Nelson seized upon this notion. ILM VFX Supervisor Ben Snow and Hickel would be involved in brainstorming sessions alongside Storyboard Artist David Lowery and our previs team. I enjoyed trading ideas with the final [vfx] vendors with whom you normally have very little interaction.

"As a result of the open approach, repurposing little beats and character vignettes became more common place. For example, an early version of Gulmira had the beat where a militant comes up behind Iron Man with a handgun only to have the bullet ricochet off Iron Man’s helmet, killing the gunman. Even after the filmmakers scrapped that version of Gulmira, Second Unit Director Phil Neilson decided to film the ricochet moment during Tony’s escape from the cave. That beat got a lot of laughs when I saw it. The Orphanage submitted an early test to the production during bidding of Iron Man in which he stands off against a tank. That bit became the cornerstone of the Gulmira sequence. In fact, when we prevised Gulmira, we just slugged in their test for that portion of it. I think the vendors really embraced this notion of collaboration. It could have been extremely politicized and difficult, but because the players were on board and open-minded, everyone 'played nice' in the sandbox."

Unlike Iron Man, the previs for Speed Racer was far more complex and divided among several companies: Digital Domain, Halon, PLF and Proof. As has already been thoroughly documented on VFXWorld, Speed Racer offered an innovative artistic experience, combining a new 2.5D live-action anime look called "Photo Anime [6]" that carried over to the immersive car racing/fighting called "Car Fu."

[7]

Previs was used as a conceptual design driving force as well as a shot and sequence design tool. All Speed Racer images from Halon and courtesy of Warner Bros. [7]

Alex Vegh of Proof was the initial previs supervisor, who worked in L.A. during concept design and early production work. He briefly worked in Berlin too, which is where the production was based. Vegh suggests that the design of Speed Racer truly pushed the boundaries of previs. "Generally, previs is used to plan how a sequence is played out or how a tricky shot is executed," Vegh explains. "The Wachowski brothers' creative vision not only challenged the traditional look and feel of previs but also its usage. Previs on Speed Racer was used as a conceptual design driving force as well as a shot and sequence design tool.

"We had a unique opportunity to do conceptual design before kicking into a pre-production/shot planning phase. The inspiration for the look and feel of the film reached out to everything from classic anime and arcade videogames to fine art photography and QuickTime vr [virtual reality] bubbles. Throughout the development process the previs team created several proof of concepts about the racetracks, the cars and the environments.

They used QuickTime vr bubbles to create the worlds the characters and cars lived in. These were executed mostly by Christov Effects and Design, under the guidance of owner Lubo Histrov, the visual effects environments art director, and leader of the in-house World Team stills department. "The idea is that one takes several high-res images from one location rotating the camera until 360 degrees of image in all directions is captured," Vegh continues. "From there the individual images are stitched together and projected on a sphere. This allows a virtual camera inside the sphere to rotate freely and translate slightly without distortion to the image projected on the sphere giving the illusion that a foreground greenscreen element is an integral part of the background. We took this idea and pushed it to the limit seeing what would happen if you distorted the sphere or intentionally got too close to the edge of the sphere with some very interesting results. Another benefit of using this technique is a camera can travel from one bubble or environment to another seamlessly like an actor traveling from one room to another. This bubble concept was meant to be the basis for how the location photography was to be done."

[8]

The track design was a synthesis of traditional 3D environments to photography projected on cards in 3D environments to complete 2D creations. [8]

The track design, he says, was a synthesis of traditional 3D environments to photography projected on cards in 3D environments to complete 2D creations. The cars themselves are traditional 3D throughout the film, while the environments were much more stylized. Inspiration came from everywhere. "We used some anime style tricks to create the race environments. Using images shot in perspective, then scaling and translating, we created environments where the camera could easily change direction, go from an interior to an exterior, move in depth and change altitude among other movements. It really harkened back to how anime artists tried to describe 3D using 2D tools. It was quite an eye opener on how ingenious anime artists were and are. Another example of this is a 'cheese wheel' road [or a 'conveyor belt']. A 'cheese wheel' road is created by projecting a road texture on a cylinder and rotating it on edge. The vehicle on the 'road' appears to be driving forward. Add slight scaling and translation of background and the illusion of movement is captured.

"The vehicles were also quite unique to the speed racer world with 360-degree turning wheels and secret weapons. In [previs world], we helped develop how these cars would move, react to each other and the world. Keeping the cars feeling properly weighted while also having them [perform] the extreme movements required of them was an interesting challenge. Using auto racing, extreme sports and even rollercoaster footage helped us develop the cars' language. Each race had its own feel or flavor. The tracks and the cars had to represent this with not just the look but also the attitude. The [Casa Cristo] Rally Race had cars with special offensive and defensive weapons. The cars in the Grand Prix were much more about the 360-degree wheel and jump jacks. Once the feel of the cars and environments were developed, we began production of the sequences. The previs animators worked hand in hand with top-notch compositors creating elements for the compositors to add 2D movement to which transformed the 3D to something quite unique and stylized."

While the look was being developed, the communications systems were also being developed. The previs team had artists who never worked together before. In order to make this feasible, Vegh says a workflow was developed that standardized how the artist worked while still allowing enough freedom for them to work efficiently. "Throughout the show, we had artists from France, Germany, San Francisco and L.A., among other places. The show itself traveled from L.A. to Germany to Chicago and back to L.A. This brought challenges of communication. We used shared screen technology amongst other resources so the artist and the directors or supervisors could communicate easily and efficiently.

[9]

The vehicles were unique to the Speed Racer world with 360-degree turning wheels and secret weapons. The vfx team developed how the cars moved and reacted to each other and their world.  [9]

"The sequences and resources were initially designed to be used as the basis for final animation. The shot design would be approved by the Wachowski brothers and from there the final vfx vendors would be delivered our 3D scenes and 2D composites. The vendors would then take our basic approved layout and bring it to completion. The idea was that the previs would not simply be a template for a final shot but the basis for the shot itself. In order to do this, we developed a scene standard so that the vendor would be able to build off of the 3D and 2D scenes that we would deliver to them. Speed Racer truly raised the creative bar on how previs and vfx are used and I am thankful that I had the opportunity to work on it and add to its creative vision."

Euisung Lee, previs concept lead for Halon, worked on a variety of critical ideas in San Francisco, which became the basis of later shot design, including Car Fu choreography concepts, simulating anime format looks in 3D and camera composition within the spherical bubbles. According to Visual Effects Supervisor John Gaeta, Lee also created some of the wildest racing shots that were extensions of action from The Matrix trilogy.

"My first involvement came during a two-minute teaser for Warner Bros. to give them an idea of the racing action," Lee recalls. "The goal was to make a somewhat videogame level animation piece of this car action. The Wachowski brothers were still working on the script, so I was brought in at ILM when Kim Libreri and Mohen Leo were still there. I worked with Colin Benoit, a previs artist also from ILM at the time. We designed a surreal version of a Hot Wheels track in a couple of months. There was no track design tool, so what I did was make a Lego-like designing tool comprised of straight and curved lines with a bend deformer. I received feedback from the brothers about adding coiling and spiraling. A lot of the ideas wound up in the film."

[10]

Halon helped with the Car Fu previs through a lot of trial and error. They tried to make abstract concepts feasible and had a lot of creative freedom. [10]

After designing tracks, Lee tested the action and shot action pieces with cars they modeled and rendered in Maya and then cut them together. Then they did more refinement and look development. "We looked at a lot of photo reference that was very sharp and contrasty with no depth of field, and a lot of anime references. We tried to figure out how far to push reality. Some of the ideas were incorporated into the script.

Another area of exploration was the Car Fu choreography. "We did a lot of trial and error concepts, some worked, some didn't, some were funny," Lee continues. "In the beginning, we debated whether or not the cars would have the same jump jacks from the series or maybe the car wheels would pop out like limbs. We were trying to make abstract concepts feasible or how to make it feasible. In the end, it was very rewarding because we had a lot of creative freedom."

Lee also came up the idea of the conveyor belt as a design tool that eventually took on greater significance. "When I was doing a piece of the teaser, I figured that it would be faster to animate cars on a conveyor belt with a bend deformer. When it expanded, it looked like a crescent moon, which is what John Gaeta called it." They discovered later on in pre-production that it was not only convenient for the animators to work on this conveyor belt but also stylistically it reminded them of anime with the cylinders of track sliding to indicate movement without actually having to draw three-dimensionally.

Lee was also instrumental in using QuickTime vr of scenic locations as a tool, which later formed the basis of the spherical bubbles. They called this HD QTVR. "We found these websites that provide full screen QuickTime vr of various locations around the world. And because I had this 24" monitor it was quite stunning to view these sites, so I bought this program that allows you to convert these QuickTime vrs into images that I can reapply in Maya to make my own vr spheres. Initially, I suggested that this could be a quick way to previs a scene in this location. And John obviously saw beyond that and realized that this could be a stylistic tool. What if we form multiple bubbles? At first, I couldn't understand it. So he was talking about onion layering multiple bubbles [providing] this pseudo parallax when you move around. Also, connecting this spherical vr together and having almost like steady cam moves along the bubbles. There were a couple of tests and I was surprised at how well it worked. The concept involved spinning the bubbles around people talking and wiping the screen." One instance occurred when Speed tells Royalton that he can't race for him and Royalton screams that he'll never race again. However, the filmmakers took the concept further by transitioning to a scene of Speed racing.

Digital Domain's Patrick Perez (Stealth), an animator and previs artist, worked out of the Venice-based studio, where the previs pipeline was set up. This pipe consisted of directly exportable rigs, animation and camera paths straight into their post pipelines. "I consider this a critical strategy in so far as the turn-around time from director approval to processing finals was halved, if not better due to the common pipeline," Gaeta concedes.

Perez focused on DD's portion: the Thunderhead and Grand Prix races. "It's funny, but before we took this on, we were thinking this could be a recipe for some craziness. But all in all, it flowed fairly well. The pipeline was pretty solid and we had [good] rigs to work with. We had our track system down, so really it was just a matter of art directing the shots. There was definitely a lot of trial and error to come up with something different, somewhere between anime and a real world situation, especially in the early days before they took off for Berlin. I was working with the Wachowskis in the office in Burbank and so there was a lot of exploring of ideas. It was actually a very cool time because they were open to input from animators and would look at what we'd do as a proof of concept test. That three-month period helped nail down how things should be moving, which, of course, was then refined during the rest of the show. Fundamentally, most of the motion aspect of what I was dealing with was fairly well hashed out.

"The central hub was in Berlin to make sure everything followed certain standards. In this situation, I was actually putting sequences together, so I would do mini-edits, different ideas for pieces of action, and would send it to them to provide context for what was happening. For instance, during the climactic Grand Prix race, there are these slalom maneuvers through half-pipes and Speed goes up on the wall and does a jump jack off some spikes. Then Larry [Wachowski] came up with the idea of Speed flipping and hitting the front of another car, which flips and gets caught on one of the spikes and then explodes. Through experimentation, one idea led to a series of progressions."

[11]

The greatest problem facing the Speed Racer team was asset management, especially since the previs was created in both Berlin and California. [11]

PLF's Robinson, a veteran of The Matrix franchise, was brought in to supervise mere days before the production left for Berlin because Vegh was unable to stay for the duration abroad. With Perez and Lee back in California, Robinson traveled to Germany with a handful of new artists. As it turned out, Robinson was responsible for coordinating the team of artists on two different continents.

"The greatest problem facing our team was asset management. During the course of the project, the cars for the movie went through continuous changes, including livery adjustments, team affiliation, race assignment and race position placement. Lead Modeler Mike Meyers and his team were responsible for making these changes. The only way to keep up with these changes, which came daily during some periods, was to use reference models. Without this technique my team would still be swapping out animation today. This made it simple to share assets with all the artists thus facilitating the ability to update shots with the latest car designs. I could simply have the latest models packed up and shipped out for the artists working back in the states and not worry about them having issues of reconstructing the animation in their scenes.

"Our second problem was the time difference. A third of our team was working nine hours behind our day in Berlin and was operating without interaction with us over the course of their workday. It was crucial to keep them in the loop with the decisions of the day, to review their work with our supervisors, Dan Glass and John Gaeta, and to assign them new marching orders. We designated meeting times at the end of our day to coincide with their mornings to keep them up to date. With this diligence, we were able to keep on top of the rigorous pace needed to produce the work for the film."

Robinson echos that working with the Wachowskis on Speed Racer differed from The Matrix in that it was far more experimental. "On Speed Racer, the brothers knew where they wanted to go but allowed us to present different methodologies on how to get there. By working with Dan and John, we were able to strike a chord with what the brothers liked and were able to create the dynamic car sequences that they were looking for."

[12]

There was plenty of action to Indiana Jones' previs, including the opening encounter with the Soviet villains in the warehouse. All Indiana Jones images ™ & © 2008 Lucasfilm Ltd. [12]

Meanwhile, working with Steven Spielberg on previs provides its own unique experience, according to Daniel Gregoire, Halon owner and previs supervisor on Indiana Jones and the Kingdom of the Crystal Skull. But then, like pal George Lucas and a handful of other directors, Spielberg truly understands the importance of previs: "I get to make the movie before I get to make the movie," Spielberg told VFXWorld backstage at the VES Awards. "The only bad thing about that is it takes about 25% of the spontaneity out of making on set discoveries because you fall in love with the previs. You don't give your imagination the chance to fly so much when you're in the practical 3D reality model of the set. So I had to fight that early, specifically by throwing out a lot of previs."

"On [Crystal Skull], we were brought on the production first, actually," Gregoire recalls. "We were second week of January, before production design, before anybody else was involved with the project -- we were in a room, working with Spielberg to start building environments that he had envisioned in the script years prior. That's an unusual situation. Steven gets on set and foundationally uses a lot of what we did as a base and then is able to completely riff off of what he is inspired by."

"But once [the art department] got going, we did receive a lot of digital model builds from them as well as building 3D models ourselves from art dept foam core models such as the Akator pyramid. They gave us things like the fancy inner chamber door, the spiral staircase, the temple top, the gear room, rocket sled room. This saved us a ton of time and got our stuff right on track with what was being built practically. We were also helping Dan Suddick with a heads up on SFX needs: Water, sand and weight issues. We did our best to make sure that the anamorphic lenses that were going to be used were faithful in our work. Things such as minimum focus distances of 3.5 feet (or something like that). Our Indiana Jones model was graciously given to us by the guys at LucasArts.

There was plenty of action to previs, including the opening encounter with the Soviet villains in the warehouse and the ensuing rocket sled ride, the Area 51 blast, the speeding jeep/swordfight sequence in the jungle, the waterfalls, the final journey inside the caves and the pyramid and final supernatural reveal.

[13]

The previs for the caves and the Akator pyramid at the end were dramatically changed onset by Spielberg. [13]

"The first stuff we worked on was the Area 51 and all the opening scenes in the desert and the warehouse from Raiders," Gregoire recounts. "That was pretty easy for us to do. There is actually a really high-res image of Area 51 on Google Earth, so we were able to build out all of Area 51 from Google Earth. I literally built the lower quarter of Nevada and then used that as a general set for almost everything in that scene, even where he crests the hill in the morning to see the Doom Town over the horizon. In previs land, where Doom Town was placed was the testing ground just west of Area 51, where you can see all the craters on Google Earth. It was just two of us, Clint Reagan [the L.A. previs supervisor] and I. The first sequence I was responsible for was the opening roadster chase, which got picked up for additional changes by [Previs Artist] Mike Comfort later in the schedule when we had nine people on the team."

Gregoire says Spielberg's "approach is to direct us and properly use us as a tool to implement his ideas. A lot of directors and productions will get us to come up with ideas and we'll toss things around, but with Steven, he actually really directs us. The effort he puts in with us is commensurate with the effort he puts in on stage, and that's why he's so successful. He uses us to the fullest extent and it empowers us to disseminate information that's useful to every department."

As for the thrilling waterfall sequence, "that entire thing was planned out to a T so that when [Visual Effects Supervisor] Pablo Helman went down to South America with a unit to shoot the plates, he knew to the altitude how high his helicopter needed to be based on the sea level altitude to get some of those shots. We were able to build those waterfalls based on Google Earth information, Google Earth topography information and then site location photo information to estimate heights, falls and positions, because Steven took those waterfalls and actually reversed them since there are only two levels of falls at this particular location. He took them over number two and number three and then took them over number one as a matter of sequence, so we were able to build the entire falls to scale, block out the entire sequence with Steven and then send Pablo to South America with a specific packet of information, so [he could perform his task more efficiently]."

[14]

Previs was used in the speeding jeep/swordfight sequence in the jungle.  [14]

However, the previs for the caves and the Akator pyramid at the end were dramatically changed on set by Spielberg. Gregoire and Previs Artist Ryan McCoy were still doing the finale in trailers with laptops until the end of shooting. "There were a lot of iterations and it was very complex," Gregoire suggests. "The final shot, where they emerge from the blow hole to observe the valley buckling and the pyramid collapsing and the debris spinning around and then the saucer rising up and disappearing and the rocks stopping and suspending and dropping and the walls collapsing and the water coming in, was 20 seconds longer in the previs shot. But it was all there. That was a long, long iterative process with Steven to figure out how he wanted to do it. He'll sit down and start sketching his own boards and they're very rudimentary, but they work. He sees it so well in his head and can draw it so that when I put it in front of an artist, it comes together when they start laying it out in Maya.

"At the top of the temple, when Indy and the others figure out the sand gag and the pile-ons rise, we had an opportunity to use ILM's Zviz [15] with a specific controller for Steven to fly around in and explore. Zviz reached a point where on [Crystal Skull] I was able to move some large sets from Maya into Zviz. ILM had written a driver, which allowed a PS2 controller to be hooked up for ease of camera navigation. It also allowed us to set keyframes for the camera by pushing the X button, create new cameras with the O button and other functions. The Akator pyramid was one place where the system worked well. I was able to hand Steven the controller and let him explore the environment setting camera in and out positions, creating new cameras and directing the scene. I was on the keyboard and mouse as a guardian angel helping him out with higher level controls and moving characters around. We then exported all the cameras and brought them back into Maya and completed the resulting shots. Even though it changed dramatically, there were a couple shots that were faithful to the angles that he chose and it gave him food for thought."

Bill Desowitz is editor of VFXWorld.