Framestore was one of only a handful of digital effects studios chosen to work on THE DARK KNIGHT by director Christopher Nolan, who notoriously prefers to work in-camera and practically as much as possible, turning to digital effects only when they offer him otherwise unachievable and un-improvable results. Framestore's shots for the film encompassed matte paintings, digital doubles (including a CG Batman), vehicles, crowds, and the highly sensitive Harvey Dent/Two-Face CG work.
Set within a year after the events of Batman Begins, Batman (Christian Bale), Lieutenant James Gordon (Gary Oldman), and new district attorney Harvey Dent (Aaron Eckhart) successfully begin to round up the criminals that plague Gotham City, until a mysterious and sadistic criminal mastermind known only as the Joker (Heath Ledger) appears in the city, creating a horrifying new wave of chaos. Batman's struggle against the Joker becomes deeply personal, forcing him to both up his game technologically and to delve deep into his own being in order to try and stop him.
Overall vfx supervision for the project was handled by Nick Davis. "Having seen the quality of the CG textures, rendering and lighting achieved by Framestore on CHILDREN OF MEN," he said, "I was convinced that they could handle the Harvey/Two-Face character. I think the quality of their work speaks for itself." He added, "Their work on the Hong Kong sequence [was also] flawless, not only in photorealism but also in the seamless manner with which it cuts with the live action footage."
Face Value
Led by VFX Supervisor Tim Webber, a team of artists and technicians worked on 226 shots for THE DARK KNIGHT (277 including IMAX and Scope versions). Some 120 of these shots involved the astonishing digital make-up created for the Harvey Dent character. Dent starts the film as a noble and good man, a politician genuinely trying to clean up his city. He suffers horribly at the hands of the Joker, grotesquely mutilated in a fire, becoming a murderous vigilante who calls himself Two-Face.
Nolan was not after the sort of look for Dent that could be achieved by adding physical make-up to his face -- indeed, quite the opposite. He wanted Dent's face to have been subtracted from by the trauma, a stripped down, pseudo-anatomical-model look that would be shocking, but in a different way to the one an audience might expect.
"The early stages of the Dent work involved quite a bit of to-ing and fro-ing between us and the production," Webber recalled, "There were a range of factors to be taken into account. It had to look real on one level, otherwise it would jar an audience out of the generally realistic vibe that Chris Nolan was giving the rest of the film. On the other hand, simply replicating real burn injuries would have just been a gross-out. We were aiming for something grotesque but not disgusting, something that enlists your sympathy whilst also taking you just up to the point where you know it couldn't be real. In addition, there were issues around the film's rating to consider. All in all, it was quite a fine line to walk. Rob Allman, our Look Dev Lead, bounced a number of ideas back and forth with their prosthetics people, and gradually a 3D maquette was developed."
Framestore modeled the 3D Dent face in Maya, tweaked it in Mudbox, and then tracked it to live-action plates in which Eckhart had performed wearing tracking markers on one side of his face. With intimate facial work like this, tracking was always going to be the toughest element. "We were basically aiming to do facial capture on set," Webber said, "And not just to get a general look, but to be pixel perfect. The work we did was new and at the edge of anything done before, I think, in terms of the way we tracked it onto the face. We used a mixture of commercial software (chiefly Movimento, tracking software designed for use with multiple cameras), in-house developed software and some new techniques we devised."
The Framestore team gathered the data they needed not only from the main cameras, but also from several digital witness cameras positioned on set, shooting at 48 fps. Individual markers could then be tracked from each witness camera, and the position in 3D space triangulated. By doing this for all the markers, the performance could be recreated in 3D. Inevitably, the data from these witness cameras was sometimes less than perfect. CG Supervisor Ben White said, "Because of the logistics, it was sometimes impossible to get many witness cameras in place -- for example, when the action was taking place in the back seat of a car."
The capture data was used to re-create Eckhart's performance on the CG model, enhancing it where necessary with an animation system that had a significant procedural component. "The key," White continued, "Was to get enough detail into our CG to give it realism. In doing so, we worked at much higher texture resolution than we normally use, and we also rendered our CG work at 4K, even for the 2K anamorphic shots."
"Besides the tracking, the other major challenge was bringing all the different surfaces and elements in Dent's face together into a convincing and harmonious whole," Webber said, "We had quite complex techniques designed into the system so that when the actor's face moved in certain ways, the muscles would respond accordingly, and the skin would slide in a particular way. Notwithstanding, there were several parts that needed to be animated by hand, including the subtle muscle motions when tensing up, and also the animation of the CG eyeball." The team also tweaked the live-action performance of the good side of Dent's face occasionally, to reflect the restrictions that the burns on the damaged side would impose.
The results of all this hard work are astonishing, a rictus-grinning demi-deathmask whose veracity you never question. "Framestore did absolutely fantastic, superlative work on Dent," Davis said, "They have achieved a memorable cinematic character and effect."
Hong Kong Guardian
Another main tranche of Framestore's work on THE DARK KNIGHT involved the creation of digital environments for a sequence set in Hong Kong. Production shot helicopter plates in Hong Kong, which Framestore augmented with CG buildings, incorporated into real plates for some shots, and into all-CG environments for others.
In his relentless pursuit of Gotham City's crime lords, Batman finds himself chasing a money launderer called Lau half way across the world. At night, Batman jumps from the top of a building and glides though the air using his cape. Shots of Batman in flight between buildings used a digital double modeled by another facility and then handed over to Framestore, who tweaked the model, executed its own animation rigging and texture work.
White said, "We needed to create two full CG skyscrapers (the IFC 1 and IFC 2 buildings) that would be seen at IMAX resolution and from close up. This presented a particular challenge, as the room interiors would be seen from such range that we had to go way beyond the use of 2D texture cards behind windows. The CG buildings team used photogrammetry techniques to help model and texture the room interiors. They then created a shader toolkit, allowing them to control the position of each office within the building and adjust the lighting, making sure there was enough variation to create a believable effect. Tiled vista plates of the surrounding city at night were shot from helicopter, and these were stitched together by the comp team to make a moving panorama into which we could place our foreground CG buildings. The sequence ended up as a combination of partial and full replacements of the IFC buildings and surrounding city, with several shots fully digital."
From atop a skyscraper, he fires sticky-bombs at an adjacent building where Lau conducts business, blowing out a window to gain access. Inside the office building, Batman engages Lau's conspirators in hand-to-hand combat. This fight scene was shot on a set, with greenscreen covering the window areas. Framestore then tracked in views of Hong Kong assembled from CG and plate elements. Cornered, Batman grabs Lau and uses another bomb to explode out the entire corner of the building. This was filmed with a model, with Framestore tracking the miniature element into its CG version of the building, and then surrounding it with a composite environment woven together from Hong Kong plate elements.
Batman lashes himself to Lau and fires his grappling gun, jettisoning an inflatable balloon that carries the pair upward. A hook-equipped C-130 plane then flies in low and extracts Batman and Lau from the sky. Framestore created a CG C-130 for most of the shots, though a real plane had been filmed it was only used in a single, wide shot. The extraction shot also featured Framestore's CG stunt doubles for Batman and Lau.
As previously mentioned, all of Framestore's Hong Kong exteriors were created at IMAX resolution, some for quite close-up shots. "We had to do very intense work on those shots," Webber said, "Down to detailing the interiors of the offices inside the building. IMAX isn't just a matter of working on a larger image -- you actually see stuff you wouldn't see in a normal shot. We had to detail everything to a much higher level."
Again, Framestore's work on the sequence elicits superlatives from vfx peers. "It's flawless," Davis said, "Not only in photorealism but also in the seamless manner with which it cuts with the live action footage."
Bats and Pieces
In addition to the major work on Dent's face and the Hong Kong environments, Framestore also created a variety of other effects and complementary work for the film, including:
--Putting digital flames to Dent's face during the incident that causes his disfigurement--A number of other explosion additions and enhancements, particularly during the hospital sequence--Crowd creation, filling the streets of Gotham City with mourners for a funeral sequence--Rig and camera vehicle removal on key car chase and crash sequence -- done at IMAX resolution--Transformation of Lake Michigan into a Caribbean location -- removal of the Chicago skyline and replacement with appropriate island elements -- done at IMAX resolution--A number of CG props, including knives used by The Joker, and sticky bombs Batman's flip-down eye-pieces, and some phone-screen inserts
Utility Belt
The Framestore team used a variety of software for their work on THE DARK KNIGHT, including Maya, XSI, Mudbox, and in-house tools for modeling and animation, PRman for rendering, and Shake or Nuke for compositing. The all-important tracking and match-move work was carried out in RealViz Matchmover and Movimento.
