Search form

Flying The Virtual Skies in 'The Aviator'

We close our focus on 3D environments with Ellen Wolffs exploration of how Rob Legato conducted the virtual aerial combat choreography in The Aviator.

Much detailed layers from the sky to the clouds to the airplanes went into recreating the aerial dogfights from Howard Hughes Hells Angels. All images © 2004 Miramax Films.

When Rob Legato signed on to The Aviator, Martin Scorseses biopic about Howard Hughes, opening Dec. 17, he knew that the director intended to evoke the 1920s-1940s moviemaking milieu in precise detail. Accuracy was key, since Scorsese has what Legato calls an encyclopedic knowledge of film history and Marty HAS every film, too! His collection includes Hughes 1930 World War I drama Hells Angels, and a key sequence in The Aviator recreates within a virtual environment what Hughes (played by Leonardo DiCaprio) went through to capture his films famed aerial combat footage. The dogfights that Hughes envisioned were so dangerous requiring 60 biplanes that several stunt pilots of that era reportedly refused to do them. Three pilots died during shooting, and Hells Angels ended up costing close to $4 million, making it the most expensive film that had yet been made. Legato, who won his visual effects Academy Award for Titanic, remarks, Hells Angels was the Titanic of its day. It was expensive; it didnt come out on time and Hughes re-shot a lot of it. People were waiting to see if he would fail.

Accuracy was the priority for Rob Legatos role as vfx supervisor and second unit director for The Aviator. He studied Hells Angels extensively in preparation for the work to be done.

Serving as second unit director as well as visual effects supervisor on The Aviator, Legato prepared by studying Hells Angels extensively. Our job was to film Howard Hughes filming in the midst of this aerial battle. That meant covering a 360° world the swooping POVs of land and sky from planes in combat. Legato began by working with digital previsualization supervisor Oliver Hotz, using Kaydara MOCAP software. He recalls, I took key scenes from Hells Angels and in 3D we essentially copied the choreography. I wanted to create the illusion that we were seeing real shots from that movie. Hells Angels had earned an Oscar nomination for best cinematography, and Legato wanted to be faithful to the films style.

We did 20 minutes of previs in less than two months, he explains. We did live input, which allowed us to do a shot of a plane flying by in realtime. These shots had the believable hitches all the things that you get when you do something `live thats difficult to imitate in a totally CG shot. It showed us how we would go from point A to point B and how we could cheat things. If we found out that a particular camera angle would cost $40,000 to actually achieve, we could makeup something else that accomplished the same thing but didnt cost as much!

Given Scorseses historical predilection, most of the effects in The Aviator were shot in-camera or with matte paintings, just like a film from that period, says Legato. But the aerial combat shots required CG models in a 3D virtual environment. Legato was careful to make the shots look like something you would see if you watched actual newsreel footage of the time, and not some impossible CG camera work. You tend to want to live with the limitations that real life would give you, he observes.

Legato and his Sony Imageworks team created a virtual 3D environment  clouds, sky and landscape poking through the clouds.

By way of example, Legato cites another project in his career when he had to recreate something that had really happened. For his Oscar-nominated work on Apollo 13, Legato says, just because I could shoot the Apollo rocket on a stage didnt mean I had the liberty to change the lighting that would have been there if I had been outside during lift-off. I had gone to the actual launch pad in Florida to check what the lighting was like out there, and when I shot the rocket model on stage I didnt vary it from that at all. That made it more real. Even when you shoot something on stage, you live with the problems that you would have had if youd shot it for real. Then when people look at it, they think, `That MUST be real, because if youd had the opportunity to change it, you would have.

The challenge with The Aviator was that they had to create both the aerial combat footage from Hells Angels and Hughes perspective as he was shooting it. Legato stresses, We wanted to create the idea and its a very subtle one that our camera photographed him filming that moment. Although we could do anything we wanted to do with computer-generated techniques, we wanted to limit ourselves to the filming techniques of the day. Even though we did this big `CG scene, I was copying what the real planes did in the take that wound up in Hells Angels. Youre seeing clouds, the sky and the landscape poking through the clouds its a virtual 3D environment much like what was there when Hughes shot his film. We emulated what was in his movie. Then we had an interpretation of what things must have looked like if you were in a third plane watching Howard film the scene. Thats the only liberty that we took with it.

Legato notes, We mimicked how this 60-plane armada flew, and it was pretty hairy. Back in 1927, Hughes built those planes. It was after the First World War and those planes didnt exist anymore, so he cobbled together planes to shoot. For us to match Hells Angels, we had to alter our planes to match what Howard did back then. Hunter Grazner built the models for this show. They had all the plans and did a beautiful job. We photographed their model to use for textures on our CG model. We created a 3D version of Hughes plane and all the biplanes.

When it came to the environment that races by beneath the Hells Angels dogfight, Legato notes, the landscape was supposed to be Germany. But Hughes had actually filmed in Chatsworth, California. Hughes choice Iverson Ranch was a popular filming location at the time, used for scores of films, including The African Queen. Since Chatsworth is now a heavily populated suburb of Los Angeles, that location would no longer work. Instead, Legato shot background plates at the relatively remote Catalina Island off the coast of Southern California. It was similar to what Chatsworth looked like in Hughes time, he says.

Legato captured footage that would provide the raw material for the virtual environment that the CG planes would fly above. Basically, I went with a VistaVision camera and panned around. Sony Imageworks created a 3D sky dome and wire frame geometry of the terrain onto which they would tile still images from the VistaVision footage. Legato observes, If you were to be exact like taking a Lidar scan of the area youd have the exact topography and youd have the photographs from any one angle that looks exactly like that piece of topography. If you lay the photograph on the geometry, you wouldnt be able to tell the difference. Using that idea, you now can spin the whole world around as long as you have enough photography to cover a given area you have basically a 3D photoreal mountain range. You can fly anywhere you want.

Re-creating believable, cloud-filled skies that appeared in Hells Angels presented another CG challenge. When Hughes was shooting the original footage for the Hells Angels dogfight, notes Legato, it took months and months of shooting and re-shooting to collect all the footage that they needed. They were on it for two or three years. When Howard didnt like the shots because there werent enough clouds in the sky and he didnt think it was exciting enough, he hired a meteorologist to predict when there would be clouds. then they would film.

Digital backgrounds were created to bring to life the tale of Hughes airplane crash in Beverly Hills.

No such problem existed when Sony Imageworks created the virtual environment for The Aviator. They did full clouds that would behave in 3D, says Legato. We originally thought that the clouds would cover so much that wed get away with murder. Because we didnt have a ton of money to do this, we thought we would use the cover of clouds to hide any problems, but we didnt need to. The guys at Sony Imageworks made it perfect so anywhere we wanted to shoot was fine. We could carve out an area without worrying whether clouds were there to cover it up.

Legato credits Sonys digital effects supervisor Pete Travers and CG supervisor Dave Sieger with making the process go smoothly. They updated something that Sony had used on The Polar Express to create clouds. But for us, they made a photoreal version. Its sort of a particle system `bed that the real cloud information was projected onto. They created, basically, the geometry that the cloud information was texture mapped on top of. Its texture-mapped in such a way that I would imagine it is in little tiny chunks, like NURBS splines. They looked very realistic.

You could view it from any angle because they broke up the real clouds into so many textures that when these pieces were reassembled they had all the texture of real clouds. My version of what they did is to imagine that you take a picture of clouds and divide it into a thousand tiny squares. Then you take those tiny pixels as texture maps that are literally sprayed onto this geometry. Basically every angle you see is of a cloud. Then they applied lighting on top of that. Its very difficult to tell how a cloud is lit. So the pictures are probably fine enough for a texture map that it creates the illusion of three-dimensional clouds. Its kind of like combining texture mapping and particle systems. The particles that were created didnt have to be that fine because theyre really just holding picture information. It probably only works really well when you do it with things that are sort of amorphous. If you chop up a picture into thousand pieces and put those pieces on top of particle geometry, each piece of that puzzle is random-looking enough that it works.

A progression from wireframe to completion of the Hells Angels sequence.

One final bit of verisimilitude in this virtual environment was the illumination of the sky. Legato explains, What we did on this film that we didnt do on Harry Potter was that as you fly around the sky dome, the sky is lit up appropriately. If youre looking towards the sun, for example, the sky is very light in color. When you shoot at a 24° angle to the sun, for example, its very deep blue. And you have all the variations in between. In Harry Potter, we didnt really do that. We had a generic sky dome.

One unique aspect to the visual effects work in The Aviator was the post-processing that Legato did to emulate the vintage Technicolor processing in Hells Angels. For the films original release, Hughes had the prints hand-colored, including an eight-minute two-strip Technicolor sequence of the films female star, Jean Harlow. Scorsese wanted to evoke that look in portions of his film, notes Legato. Way back in 1927 the only color process that was available was two-strip Technicolor. So you could only use red and green. If you shot a blue sky, the blue sky would always be this cyan green, and it tended to look artificial and not like real life. Martys version of how he was going to tell the story is that if he were filming Howard Hughes at the time and wanted to use color, hed be limited by two-strip Technicolor. So thats what he chose to do.

More shots from the stunning crash in Beverly Hills.

Legato worked with Technicolor to break down the process in a way that could be emulated digitally. Recreating it was a lot of work. I went through the history of Technicolor, imitating what they did in the lab. We basically printed out three-color strips on transparency material that we could hold up to the light. When we took one away, we could understand what the two-strip look was. I was so blown-away by the cleverness back then - that somebody could come up with the idea of taking three pieces of black-and-white film and put the appropriate color filters on them. Its still black-and-white except that the colors record differently on the different black-and-white and gray layers. Then you combine them together with ink and you create a full color picture. Its like a magic trick. Not many people in America would know what that look was, but Marty knows. So I matched it as closely as possible.

Looking back on the painstaking attention to detail that went into The Aviator, Legato admits, I adored everything about it. He cites the aerial battle in the virtual environment as probably the sequence that Im most proud of. Every shot in the sequence looks pretty believable, like the real thing at least as close as we could get.

Ellen Wolff is a Southern California-based writer whose articles have appeared in publications such as Daily Variety, Millimeter, Animation Magazine, Video Systems and the Website CreativePlanet.com. Her areas of special interest are computer animation and digital visual effects.