<em>Speed Racer</em> and the Art of 'Photo Anime'

[1]

Speed Racer offers a new kind of digital cinema experience called "Photo Anime." All images courtesy of Warner Bros.  [1]

The goal of Speed Racer [2] was to re-imagine the stylistic language of anime in HD with spherical 2.5D photo elements known as "bubbles" and super saturated colors to achieve a new kind of digital cinema experience called "Photo Anime." This obviously required a lot of artistic decision making and technical ingenuity.

Two key members that were instrumental in helping to create the Photo Anime look were Darren Poe and Jake Morrison. Poe, the head of Digital Domain's compositing team (We Own the Night, Pirates of the Caribbean: At World's End [3], Charlie and the Chocolate Factory [4]), assisted John Gaeta [5], one of the lead visual effects supervisors, during early HD proof of concept testing, "which pushed ideas of mashing HD performance and 360° spherical background layers into techno pop, multi planar scenes and then later on in post he helped us craft a color and light strategy for CG cars, which was really brought to fruition by his 3D composite finesse techniques. He is perhaps the best applier of 3D composite technologies found in Nuke," Gaeta commends. "He was masterful at experimental and designer focus, motion blur, textural 'look' (grain free, noise free, beautiful and scrubbed, super clean) and 'Techno Color' enhancements, allowing us to get close to the idea of a Photo Anime feel."

Meanwhile, Morrison (digital effects supervisor on 300 [6] and CG supervisor on The Matrix Reloaded and The Matrix Revolutions [7]), who also worked closely with Gaeta, spearheaded the in-house compositing team called Exhaust. He not only participated at the front end with HD proofs, but was also the top executor of making whole scenes with designer "Faux Lenses" out of bubbles and HD. "He did hundreds of shots, which became the nuts-and-bolts stylistic drama inventory -- essentially our Pop Art version of Sin City [8]," Gaeta adds.

For Poe, the early 20-shot proof of concept test was invaluable in figuring out with the Wachowskis how the stylized look of Speed Racer would work in HD: "Basically, they booked out some studio stage space [here in L.A.] and hired some actors and shot them on greenscreen and did some realtime compositing. I was pulling their HD footage using a codex, which is an HD realtime capture system. I was directly wired into that using Nuke as well, so I was pulling the footage as they shot it over to a workstation and taking these spherical bubble backgrounds that they had shot doing comp experiments on the fly: color and image scrubbing and multi-planar treatments. So I sat with them as they were shooting that stuff. The brothers went back and edited a sequence together and delivered that footage to us here at DD.

"And then working with a matte painter and building a multi-level 3D bubble environment of this city behind a penthouse and putting things on cards, which would normally be 3D objects in the scene but having them not have proper parallax. And finding a way to mix that with the live-action stuff to have it look kind of right but still have the look of anime. So, it was basically an experiment to see where the interface of 3D and 2D needed to be for the footage to be able to exist in this world without looking totally bizarre."

They also experimented with how to treat the color to get the super saturated look that they wanted back on film. Poe says that was quite a challenge. "We had to be mindful of the gamut of film but really pushed the color saturation as far as we could and did a lot of profiling of film stocks and HD material and plotting out the curves and figuring out how to enhance the look. The goal in the early test was to find a piece of film that looked almost identical to HD footage. Film has some limitations compared to HD in terms of color saturation. There were some areas where we could never get the film to look exactly like the HD, but we did a lot of mathematical plotting and analysis of inverse transforms of HD and film color matrix to try and get a 1:1 match."

[9]

Darren Poe and Jake Morrison experimented with how to treat the color to get the super saturated look that they wanted for the film.  [9]

After testing, the Wachowskis were sold on shooting in HD with the Sony F23 camera, which offered, among other things, great depth of field and superb color saturation.

"I went to Berlin in August toward the end of the shoot, and did something similar to what I did in L.A., which was take the cuts that they had shot, work with the matte painter there doing the stylized bubble paintings and also their previs artists, and put together test comps of different types of footage," Poe continues. "One of the big focuses there was how to take the non-greenscreen footage and treat that so it could co-exist with the 2-1/2D bubbles. I would take things that they shot and then go in and roto and re-paint sections of it and split it out into multiple layers and give things slight drifts against each other and post multi-planar camera moves to give it that anime vibe."

Poe did some imaging scrubbing experiments too. The goal of that, of course, was to have a completely grain-free movie inspired by fashion photography as well as anime. "We looked at super airbrushed makeup ads and tried to find ways procedurally to get that look without labor intensive work for compositors later on. For that effect, we used a combination of Furnace tools for de-graining and found ways to isolate contrasty areas of skin tones and minimized contrast using color corrections and added detail back in. There was no film grain but a certain noise level needed to be addressed, so we experimented with getting rid of high frequency noise and homogenizing color within individual objects to give it a more idealized version of the image.

[10]

One of the big challenges facing the vfx team was starting racing scene sequences as 3D previs which would then became full 3D animations and 3D environments.  [10]

"One of the big challenges at DD was our sequences where racing scenes started out as 3D previs and then became full 3D animations and 3D environments. And so what we came up with were ways to step back from 3D to 2-1/2D. We were then re-projecting the 3D environments onto layers of bubbles and then going back and compositing new camera moves and drifts between the plates. Our big challenge was to make the inherently 3D stuff have the same consistency of the work being done at other facilities in the anime-like vein. Maintaining consistency was the evolving stylistic direction of the show across multiple continents.

"And we developed a stylized de-focus treatment for different scenes where we used arbitrary shapes to make it look more graphic. We'd have a square de-focus or a sharp circle, overlapping shapes. We had various iterations of that to look at here. They wanted the cars to reflect their environments but asked to take it up a notch. For example, to have streaks of light, we built HDRI light tunnels that the cars would drive through, to enhance the sense of speed and give it some graphic line treatments. We looked at a lot of commercials and music videos for inspiration. We built a pipeline using our image-based rendering here at DD that would allow us in the composites to independently change the color of the environment, the base color of the car and all of the CG lighting and reflections. For example, a blue car would only reflect a more saturated blue light and it would still have the correct light modeling response -- it was not just a global color. In other words, the lighting of the car would be treated as just blue but the reflections would be the natural environments. This would allow the colors to pop.

"For me, it was interesting because it's not something you would normally do on a project. As a compositor, typically here at DD, we mostly focus on photorealistic work. But on Speed Racer, we had to unlearn what you've learned over the years of what makes a good composite in terms of the integration of elements. You are forced to think about the individual shots as if they were drawn as cel animation, except, in this case, you were using photorealistic imagery on the different layers instead of illustrations."

[11]

A palette was chosen for each sequence. The Grand Prix has extreme saturated blues and hot whites, while the Thunderhead Pass features a more golden hour with stronger yellows and reds. [11]

In constructing the Exhaust compositing team on the fly in Chicago, where the Wachowskis hail, Morrison and his team were responsible for building the tools and the infrastructure. Wife Amanda Morrison was the lead compositor. "We started as a supra vfx editorial department doing the directors' cut of the film and then came back to L.A. to do post.

"Funnily enough, this is exactly what I did on 300 for Warner Bros. But visually it was at the opposite extreme. You can categorize it in terms of focus. This never fell where you would go with a true camera. We either went way overboard to bring the audience in or we kept everything tack sharp to let everyone see the vistas. There's a sequence called the Driver's Club, which contains huge wood panels and is very opulent browns and reds. It's very Versailles with massive windows on the side and a gigantic swimming pool with synchronized swimmers, with the blues punching through the browns and reds. So you start with everything tack sharp where the focal point is allowed to be infinite so you can take in everything. And then as we move through the sequence, there's a gathering of people in the center of the room, where [legendary driver] Cannonball Taylor, [Speed's adversary] Royalton and the family all come together, and we pull the focus in tighter and tighter where everything behind these guys is totally out of focus. As part of that, we had another choice, which was the 'circle of confusion.' We chose diamonds as an impossible diaphragm for the camera to add a slightly subconscious feeling of more wealth to the room."

Morrison adds that there were other shots, much like the Hitchcock dolly zoom, where you make a particular character feel unsteady. They took different layers and scaled and shifting them around in "impossible" ways. They might bring the background in and push the foreground away and take a mid-level and do something more extreme than either of the first two. But the language is already well described in anime, so, he says, you're not creating anything brand new stylistically. The trick was bringing it into live action without totally overwhelming the viewer and making it look too synthetic.

"It's very vibrant and super saturated, but there's always a theme running. It's very clear that there's a palette chosen for each sequence. In terms of the races, the Grand Prix has extreme saturated blues and hot whites, but there are amazing sequences that are bold and different. Thunderhead Pass tends to go for a more golden hour with stronger yellows and reds, then very clean blues punching through that. The school section at the beginning, when Rex picks up young Speed from school, contains the largest cacophony of all. That's basically taking the outside of the hue wheel, all of the saturated colors that live on the side of that, and sprinkling different advertisements and flowers. In fact, in an odd way, the less saturated stuff in that makes it a more unified palette."

Morrison admits that the bubbles were very useful in manipulating the live action, splitting them into layers as though you were working on a rostrum table. "It was very much an anime thing to do but it goes back much further than that with matte painting. One good example was in a hotel room during the Casa Cristo rally sequence. Taejo [12] and Horuko are having a conversation and the camera is wheeling around from right to left on Taejo and left to right on Horuko. But what we've done is taken the bubble and overridden the camera move and let the whole thing just spin at a consistent rate while still swapping backwards and forwards to frame up on the nicest bit of the room. The bubbles gave us that kind of functionality. Depth of field was big in terms of making sure people got enough stuff to feel the right way and then using the circle of confusion to enhance the feel of that. If you're in a rainy scene, the circle of confusion would be a vertically stretched lozenge, which is in sympathy with the rain falling down, and very much controlling the background to help the audience feel slightly uncertain."

Like Poe, Morrison maintains that the trickiest part for artist and supervisor alike was to be counter-intuitive to their previous photoreal experiences in compositing. "To some degree, for a certain look that we developed in Berlin and Chicago, we then had to tell people when to go off the rails and when it was not the case to do that. That was the hardest thing for an artist working on this show. It's not empirical, there's no yardstick to measure everything against. It's a series of aesthetic decisions that, only by doing it for a reasonable amount of time, and working with the brothers, do you get a feel for it."

Bill Desowitz is editor of VFXWorld.