Search form

'Harry Potter' Goes Naturalistic: Part 1

Harry Potter and the Half-Blood Prince looks a lot more photoreal, and Bill Desowitz finds out how and why from Tim Burke, Double Negative, MPC and Rising Sun Pictures.

In the opening that was re-rendered by Double Negative for IMAX 3-D, Diagon Alley is attacked by the Death Eaters. All Images: © 2009 Warner Bros. Ent. Inc.

View clips and trailers of Harry Potter and the Half-Blood Prince at "Harry Potter Haven" on AWNtv!

Like most franchises these days, Harry Potter has become more and more naturalistic in its vfx. That was the mandate of director David Yates, who returns for his second outing with The Half-Blood Prince. There's no denying the influence of Chris Nolan's grittier take on Batman. As Harry (Daniel Radcliffe) rapidly matures into manhood under Professor Dumbledore's (Michael Gambon) wise tutelage, and the Death Eaters wreak havoc on both the Muggle and Wizard worlds, there is a wonderful sense of foreboding that permeates the movie. After all, this is the transitional story as Harry and his pals prepare for their final confrontation with the Dark Lord in The Deathly Hallows, which has been split into two movies for 2010 and 2011.

And it's paid off for Warner Bros. at the box office, where Half-Blood Prince has broken the record for the biggest five-day gross in history, making $158 million.

Overseeing the vfx once again is Tim Burke, who relied on Double Negative, MPC, ILM, Rising Sun and Cinesite to provide most of the work, which has gotten a lot more sophisticated and complex in keeping with the escalating jeopardy.

"David treats Half-Blood Prince very much as a drama piece," Burke explains. "The way we worked the team was not to get bogged down in visual effects set pieces. And a lot of the work throughout the film is invisible: a lot of environments and set extensions and all of the bluescreen work that we hope people won’t spot that add to the storytelling. As much as possible, we allowed him to shoot things freely and pretty much work with the actors and worked around him. There are only a couple of really big set pieces. A couple of visual effects-led title sequences with CG environments and we relied on previs. But we let the drama unfold."

And the franchise has taken particular advantage of the latest advancements in HDRI, facial capture and fire and water simulation. "It's been interesting seeing the companies improve that have worked from the very beginning," Burke continues. "I think the environments are an important thing that we're doing more and more of, and, as the tools they develop get more sophisticated and photorealistic, that's pretty much the key. We feel confident that we can shoot on partial sets or complete greenscreen stages, knowing that we can make a photorealistic environment for the actors to be in later on. On these films, like a lot of franchises, there's a lot less traveling to locations now and more work being done back here at the studio. I think we had something like 15 sets built on the backlot at some time or another, of which have partial or total bluescreens.

Yates wanted special attention paid to the crossover from the Wizard world to the Muggle world to emphasize the death and destruction at stake, with Double Negative rebuilding a large portion of London's West End in CG. As an added bonus, the opening and closing will appear in IMAX 3-D beginning July 29.

Double Negative also animated the destruction of the Millennium Bridge using dnDynamite and dnOcean with a little Houdini on top of the CG water.


For complicated cloudshapes, Double Negative tweaked its proprietary volumetric rendering toolset, DNB, which is a combination of Maya fluids and DN Squirt.

"David wanted to open the film with a huge, blockbuster sequence and really kick start the film," Burke adds. "In the original book, there was the idea of the wizards attacking and invading the Muggle world, and he wanted to illustrate that quite efficiently, almost as an act of vandalism. So we came up with the idea of The Dark Mark appearing above the city of London. From there, Death Eaters race down, fly over Trafalgar Square, then try to cross over to the entrance of The Squeaky Cauldron, where we know that Diagon Alley is secretly hidden, and that is where you enter the Wizarding world. And once there, they attack Ollivander's Wand Shop and kidnap him. So the whole thing was conceptualized as one continuous move. David wanted it to be a real roller coaster, breakneck race down through the streets. And the only way we could do that was completely digital and we turned to Paul and his team because of the recent tools they developed on The Dark Knight. Obviously we were able to photograph all of these locations so that we could recreate them as accurately as possible."

"Because we were in a true 3-D environment, we could create the correct stereo separation," suggests Paul Franklin, Double Negative's visual effects supervisor. "This was certainly my first go at doing true stereo 3-D. I really think it's going to be a significant force in the future of cinema."

For Half-Blood Prince, the 3-D focused on the titles and the opening. "We travel through this very dimly lit cloudscape and we created that with our proprietary volumetric rendering toolset, DNB, which is a combination of Maya fluids and DN Squirt, which is our in-house fluid toolset, which we actually used to form the shape of the clouds and then rendered them with DNB," Franklin explains. "And there were some other tricks to give us a sense of lighting within the clouds, first revealing the Warner Bros. logo and then the Harry Potter logo coming through. It's a very architectural set up and then as you begin to push through this, the lightning flashes are timed with the soundtrack, which sounds like a distant war, which relates to an underlying theme of all-out war. Eventually this pulls us through to modern-day London, which we actually shot on location at the building of the Great London Assembly. It's a very ultra-modernist building very much at odds with the Wizarding world of Harry Potter, which always has that wistful, Dickensian look to it. David Yates has liked using modernist architecture for depicting the Muggle world. An atrium of clouds builds up with a very strong architectural sense. The huge Dark Mark falls over London. That was a real achievement using DN Squirt. We wrote a few new tricks and tools for it that allowed us to work with the underlying geometry of the Dark Mark, particularly a huge scull with giant fangs, and basically fill it with clouds. But then as we're animating it, we're getting all of the edges tearing away and bleeding away. It's a balance between natural cloud forms and the stylistic shape of this skull. We wanted it to look unnatural."

For this entirely CG sequence, Double Negative spent a lot of time taking aerial stills of downtown London. "We worked with a lot of satellite photography that we got off the internet," Franklin offers. "We got a hold of street plans. And we created a very detailed 3D model of this section of London and then populated it with CG people and vehicles. We developed a new system using Houdini to procedurally model all of the trees. From my point of view, the route that the camera flies in is pretty much the same one that I walk from Charing Cross Station into Double Negative every morning. One of the things about Diagon Alley is that everything had to be carefully hand-made. We took the components of the set usually to extend it based on our imaginations.

"After the Death Eaters grab Ollivander, we cut to the Millennium Bridge. This is a fantastic horizontal suspension bridge outside of the Tate Modern Gallery on the banks of the Thames. We're back in the modern world of steel and glass. The ominous sky is a matte painting derived from a plate that was shot off the roof of Double Negative, with lightning flashes added to it. The three plumes of the Death Eaters descend on the Millennium Bridge and twist around it flying back and forth. They vanish but they cause the bridge to turn and twist and ripple. We switch to a fully CG bridge and replace the river with a CG river using the dnOcean toolset so that we can create waves and swirling tracks left by the Death Eaters. We shot some close-ups of people on greenscreen. We then created the London environment as a 3D reprojected painting behind that. The bridge begins to break up and we treated it effectively like a character animation and rigged it using our character toolset because the deformations were so extreme. But then the character animation of the bridge drives a secondary dynamic system, which is built using our dnDynamite toolset, and this causes things like the decking planks to pop away and eventually as it really breaks up, we see the suspension cables snaking and whipping away, also using dnDynamite. Eventually, the bridge snaps in two and they originally had animations of people falling into the river but that was deemed to be [too intense]. We were able to animate proper water surfaces using Squirt and some enhancements to Houdini on the top of it."

Director Yates requested a much simpler Pensieve design so that it becomes much more of a surprise as you get into the memory.

According to Franklin, the idea for this sequence came as a result of Yates having a dream of the Millennium Bridge collapsing. It departs from the book by literally attacking the Muggle world and puts Harry Potter into the real world.

Double Negative worked on two other major sequences: the attack on the Burrow, which is also new to the film, and the Pensieve or Memory Bucket. "The Death Eaters attack the Weasley house in a new scene where we did quite a bit of fluid dynamics to create a wall of fire," Franklin continues. "Now it takes place in open marshland as opposed to the wooded area depicted in earlier films. My intention was to make it look like a localized firestorm appearing around the house. There's a fire snake again and we relied on a character animation approach while tweaking the Squirt toolset to get the appropriate movement.

Of course, the Ponsieve is the vessel through which memories are reviewed. "Yates required a much simpler design [this time] so that it becomes much more of a surprise as you get into it," Franklin suggests. "We made a very shallow, parabolic dish, which floats suspended in the air. This is activated by a vile of memory fluid. As the memory fluid is poured into the dish, it becomes a swirling cloud shape with strands of falling ink. It's almost like words on a printed page coalescing out of the ink. It was done in Squirt. We also used live-action elements by dropping ink into an aquarium."

Meanwhile, MPC specialized in both the Quidditch trials and the match. This is definitely something that has improved throughout the franchise. "I think Quidditch is a way of monitoring the change in computer graphics on Potter through the years," Burke says. "Perhaps it didn't look great in the early years and now we're getting a slightly more photorealistic look to the game play and the movement and digital doubles and environments. And we made a big effort on this one. David likened them to rugby matches in school on a wet, autumn day, in the mud -- that real, soaking wet, sodden environment. The first thing we thought we'd do is pay closer attention to the environment, including the stadium itself because I don't think enough attention has been paid to that previously. People have tried to get some exciting game play but not really help themselves to the way the environment affects the speed and the action of the camera movements. So we redesigned the environment and the stadium a little bit. We added more towers to give more of a backdrop to fly against and we created a much closer environment by introducing new mountains and a huge waterfall around the arena, so the stadium sat more in the middle of the bowl. This way by putting players against mountains or the stadium rather than the sky, they created a greater sense of speed. And MPC used a texture library of shots made in Scotland mapped them all together at a very high resolution to give us a complete and unrestricted world for flying. The trials were wet, misty, overcast day, and the match is played in snow for a different feel."

According to Nicolas Aithadi, MPC's visual effects supervisor, they reviewed aerial shows and motorbike races and tried to understand how to make Quidditch look more real. "One thing we found out was the advantage of using different lenses," Aithadi explains. "We used long lenses to flatten everything and have the camera catch up with the action. The other thing was to make it more exciting, there had to be more CG involved, so we had to find a way of handling all of those close-ups. What we did was we tried what we called videogrammetry, where we photographed the actor on a chair with four cameras. We had 68 tracking markers on the face and the riggers animated the CG face at the same time we were capturing the textures. The markers were small but we were able to get very good deformations in the face and textures. We used a very flat lighting environment when shooting to make sure we could use normal textures. This gave us something very real because the texture wasn't painted. It was straight from the film. We created blends to blend the four cameras and flattened them in UV space to get UV textures that were then sent to a shader. The CG work was so good, in fact, that it didn't require live -action takeovers.

MPC did a lot more intensive environmental work for both the Quidditch trials (here) and the snowy match.

Full CG clothes so close to the camera required a lot more resolution. For some shots they had live-action actors without capes so they extended the capes later in CG. Between the trials and the match they had three different types of material to deal with.

"Trying to simulate the cloth on fast moving characters wasn't really realistic," Aithadi adds, "so we decided to remove the translation on the Quidditch players and simulate the cloth at the origin; then we were using winds in Maya to simulate the translation."

Environments, of course, were full CG. MPC created panoramas from the production stills and created a 3D geometry using the trees in the photographs to determine scale of the mountains around the stadium. It had to be 3D in order to get proper parallax and proper depth. Hazing and clouds were added later by the compositing team. For the match, the original environment used in the tryout sequence was snowed up and MPC used particle systems to create the falling snow.

Rising Sun had its biggest contribution yet to Harry Potter, delivering nearly 300 shots. The most important work was the introduction of Professor Slughorn (Jim Broadbent) and the transformation of his chair (into his human form) and room. "There are a lot of props, including a grand piano and a chandelier and grandfather clock," offers Gregory Yepes, Rising Sun's visual effects supervisor. "It's all broken apart when Harry and Dumbledore enter the room because Slughorn is hiding from the Death Eaters. Dumbledore realizes it's a spell and counters the spell to straighten everything out. Our people did some really clever things for this sequence. For instance, there's a picture frame on the wall and it tilts to straighten itself out. That's the only moving practical thing on the set."

Everything else moving in the sequence is CG. Rising Sun used Maya for animation and Houdini for vfx and composited in Shake (they have since started using Nuke).

Rising Sun Pictures got to play with the Slughorn transformation scene, which involved a lot of clever fun with CG props.

"Tim and David Yates gave us the rough boundaries for the sequence and we just played around with it," adds Yepes. "It was such an overwhelming amount of objects, but we got great reference from Tim and his team of digital pictures and a catalog of every object in the room. For some of the bigger pieces, we got them cyber scanned. We also had a lighter on set to make the tracking and camera matchmove done. We modeled low-res versions of every prop in the room so that when we started doing animation layout, we picked from a catalogue and per a designated camera move. That was carefully driven by Tim and David. The whole point was to avoid spending a lot of weeks working on models and textures. There were a lot of layers, especially debris."

Bill Desowitz is senior editor of AWN and VFXWorld.

Bill Desowitz's picture

Bill Desowitz, former editor of VFXWorld, is currently the Crafts Editor of IndieWire.