Likewise, the lighting of the head was linked to the set lighting. "That was the philosophy," Litt continues. "Brad drives the performance and the director and DP drive the look and we hold true to that. In other words, it's 100% set capture of the environment and the application of that environment in the renderer. This was a pretty unique job for the lighters. It's an evolutionary step for the industry in terms of HDR capture. Like animation, we built a system that was intuitive and controllable, but all physically-based. Our tracking team did an amazing job of capturing the minute detail of every single lighting setup. It was a full survey of every lighting setup: photographic and 3D positions. They surveyed around 80-100 lighting setups with a CG head. That data then fed into the new pipeline built for the show."

Tight collaboration, therefore, was crucial between compositing and lighting. "A lot of the setup was done in Nuke, which, as everyone knows, is very powerful in both 2D and 3D," Litt suggests. "Teams of compositors would be the first to touch the shots by ingesting this data and bring it up in Nuke. We started off by using straight HDR spheres but slowly came to realize that it wasn't getting us the subtleties we needed in terms of localization of lighting. With a head moving around in a room or gradations of light across the head itself, some of the light sources are very close. There's a candlelit scene and certain types of falloff that you don't get with an infinite sphere, so we developed a hybrid system of taking all the data, mapping some of it onto a sphere and then reconstructing all that geometry and all the bounce cards and light cards and lighting sources and taking high dynamic range textures, putting them on all of that data and then, using custom shaders all in mental ray, driving the lighting from this combination of the HDR sphere and HDR set geometry and the 3D position of the head. Even the HDR sphere went through a new process that was developed in Nuke for this show, which we call 'Repositioned HDR' or 'Repo HDR.' In Nuke, they could import the camera, the head position and all of that and generate animated HDRs as the head would move around. It was a new way of lighting that worked really well in terms of volume. The head is all over the map; it's 325 shots with different lighting setups because of his movement within a scene. Knowing the schedule of the show and the ability and time it takes to match lighting by hand, you're never going to get all the way there. This allowed us to push through a lot of shots once the lighting pipeline got rolling."

There were tracking breakthroughs on Button. For such intricate tracking, Digital Domain made sure it could synchronize and time-code every frame. Courtesy of Digital Domain.

And there were tracking and compositing breakthroughs as well. For such intricate tracking, Digital Domain made sure it could synchronize and time-code every frame.

"In the post process, we track the main camera as well as the high-end HD cameras to re-triangulate the performer on set," explains Tracking Supervisor Marco Maldonado. "We reorient to the survey, which is the onset environment scaled 1:1 to real world numbers. And once we figure out where that's going to be within a 3D package, that's embedded to some degree and everything else has to fall within the survey. Once we have him in 3D space, we take that point cloud to drive the CG head and CG clavicles. So there was always one person to track at a time and different body types to track with the CG Brad head. Every actor had its own set of challenges because we have to remap the point cloud to each performer. A lot of the tools for our tracking software were re-written to help with the placement of the point cloud."

The main difference on Button is that compositing actually played the very first part in the lighting pipeline. "We would take these HDRs, make sure everything was correct and reposition them to where Brad was in a scene, run that out and that would be the first pass, which would go off to the lighters, who would then finesse the technical set up of the HDR," adds Compositing Supervisor Paul Lambert.

The tugboat was shot on a greenscreen stage and its journey had to be seamlessly integrated into full digital environments that traversed rivers, and oceans environments. Courtesy of Asylum. 

"But we had a pretty new drill at the beginning of the pipeline and part of it was getting the grades into place, like how David wanted the scenes to look," adds Compositing Supervisor Janelle Croshaw. "We were setting the color and the tone. Nuke is the backbone of everything here, so every single department ended up having a check, a script or a comp. An image sequence was basically an auto comp of the head and we had track check, anim check, roto check. We figured out on Zodiac that digital data can get really messy and there can be a lot of it. And so we worked on automating on Benjamin, so everyone could see it immediately. We were able to automate the process going out as well and reverse all the grading we did because David likes to do all the grading in sequences up in DI with the shots that don't have visual effects. It was important that tracking needed a good way to view what they were doing over the plates, so we set up track tracks where Marco and his team could hit a button out of Maya and it would run an automated Nuke script that would grab the graded plates and the roto and do mini-comps so they could see how their tracked head fit within the shot. So it helped all the departments a lot."