<em>Angels & Demons</em>: Vatican VFX on Steroids

[1]

Double Negative reproduced St. Peter's Square using proprietary photogrammetry software and satellite imagery © 2009 Columbia Pictures Industries, Inc. All Rights Reserved. Courtesy of Double Negative.

"It was Da Vinci Code [2] on steroids," laughs Angus Bickerton, the overall visual effects supervisor of the follow-up, Angels & Demons (from Columbia Pictures), which pits Tom Hanks' professor Robert Langdon against the murderous Illuminatti, an ancient society that is preparing a large-scale disaster.

The script called for several key sequences to take place in Rome, in churches or cathedrals that the Roman Catholic Church denied the filmmakers access to shoot in. Thus, it was thus no easy task to gather photographic references of the various locations. The teams had literally to pose as tourists and shoot as many photographs as they could in the most genuine way. Clearly not the best process when one plans to rebuild the locations via photogrammetry…

The first artist to defy the Roman Catholic authorities was VFX DP Stefan Lange, who gathered a huge library of stills for the preproduction team to work with. "We used these as a resource -- creating QTVRs that could be virtually navigated -- for screenwriter Akiva Goldsman," Bickerton says. "We had less than six months to deliver an estimated 800 shot vfx count. VFX Producer Barrie Hemsley [3] was the genius who shepherded the tallies and budget with consummate ease. We started working on shots immediately, even though this might mean some of these shots may ultimately be cut. It's always hard to tell a creative artist that their endeavors have just been excised from the film, but the benefit is that creative dialogue about sequences can start from day one of post. I think it's invaluable to block in vfx as soon as possible to help the editing process, and we had the whole film temp composited by an in-house unit two months into post. In the end, we worked on 1157 shots, but only 917 are in the final cut: DNEG did 359 shots, CIS Vancouver 240 shots, MPC 201 shots, The Senate 196 shots and our four-man in-house unit 161 shots."

One of the major concerns was how the team would track handheld shots as the cameras moved through the many crowd shots, often meaning that markers on greenscreens were completely hidden. In the process of investigating potential systems for on set mix and overlay of low-resolution set extensions, 2D3's system set up by for Sweeney Todd [4] was reviewed. It used a small scientific analysis camera mounted on an arm above the camera. Coded markers were posted on the green screen and an initial survey using a digital stills camera was tracked in Boujou to establish positions. A realtime PC system connected to the camera then tracked the markers in realtime.

Bickerton found that the only disadvantage of the system was that a desktop PC had to be cabled to the camera, which didn't fit with [Director] Ron Howard's shooting style. "The new breed of AVCHD mini camcorders were the solution. Nine Canon HF10 cameras recording a very compressed 1920x1080 24P were purchased, along with Sony wide angle adaptors. They were then mounted on top of each 35 mm camera and Double Negative devised a marker system on the slates that would allow the HD camera's offset to be calculated. Whenever a longer lens was used or markers were hidden by foreground crowd, the HD cameras were used for tracking."

Recreating an Architectural Wonder
Production Designer Allan Cameron built sections of St. Peter's Square and the Basilica at Hollywood Park, but due the massive size of the location, the sets were built at a smaller scale. "This meant that Double Negative had to build two Piazza environments, one was actual size for full background replacements and the other matched to our hybrid set," Bickerton notes.

Led by Visual Effects Supervisor Ryan Cook [6], VFX Producer Fay McConkey, CG Supervisor Graham Jack and 2D Supervisor Victor Wade, the team at Double Negative set out to reproduce the famed St. Peter's Square. Since LIDAR scanning of the set was impossible, the layout and dimensions of the location were created using proprietary photogrammetry software and satellite imagery. "St Peter's Square created new challenges compared to our previous environment work where we mostly had skyscrapers and office blocks," recalls Cook, "The Vatican is almost entirely intricate buildings, statues and delicate detailing. Because it was a daytime build, it was also particularly unforgiving. In additions, hundreds of props were modeled that could be used whenever required, such as dustbins, souvenir stands, flags, etc."

The lighting created a major challenge as the location was lit by a very complex lighting environment. After the team had added everything in, there were several hundred light sources in the scene, which, even when they had optimized, meant the shadow generation was taking far too long to calculate. It was decided, therefore, to use RenderMan [7]'s point cloud baking tools to bake all the illumination into a series of point clouds. Using additional proprietary tools, the team was able to combine all the point clouds for the different groups of lights together, so that they had one point cloud for each region. By splitting up the lighting in this way, they were able to use memory intensive techniques like ray tracing to get the best possible quality in the lighting.

They were also able to use the tools that Pixar had added to calculate ambient occlusion and color bleeding from the point clouds. These were then combined together in the renders, but could be overridden if the lighting had to change. Baking the lighting in meant that the lighting artists could build their scene, hit render, and immediately have a decent starting point for realistic lighting. It also allowed the team to do global illumination effects and produce light bouncing from building to building. "Without the baking technique, the complexity of the models would have been way beyond what we could have managed," adds Jack.

New Crowd System
One of the biggest issues for these scenes was the immense crowds. For the majority of shots, where no specific action was required, the crowd was created via live action footage being projected on cards. The team shot extras in groups of varying sizes, using as much variation as possible. For the more complicated crowd shots, Double Negative developed a motion capture-based crowd solution built around Houdini [9]. First, the team captured a large variety of animation clips, which were then gathered in a motion library. Then, the R&D team wrote a solution inside of Houdini that placed crowd's agents as particles and layered animation from the motion capture clips on top of that initial placement and/or motion. Through a series of rules and based on the actions needed for the shot, the team ran the particles through a simulation that drove the application of the required motion-capture clips. Based on the speed and direction of a given particle, they were then able to apply the correctly blended motion onto the particle, so that the motion of the agent matched the motion of the particle. Houdini CHOPS network was used to blend between multiple motion capture clips to apply animation to each particle, which resulted in a smooth blend of separate animation clips.

In addition to the locomotion system, the crowds system was also capable of triggering specific actions within a simulation. "The key difference with most off-the-shelf crowd software packages," Jack continues, "is that the particles in our crowds simulation drive the application of the animation data, whereas in other systems, the animation motion drives the gross crowd motion."

Double Negative also tackled the climactic anti-matter explosion above the Italian capital. The explosion concept progressively changed from being a traditional nuclear explosion to more of a nebula effect. Recalls Cook, "We needed to create something that looked like the astronomical photographs and satellite images, but it needed to explode and evolve though time. In-house concept artist Christoph Unger worked on some concepts of what the explosion would look like at various stages."

When the bomb goes off at first, it starts like a traditional fireball explosion, for which practical elements were used. The fireball then contracts into a singularity, so all of the flames and smoke start to swirl back into the centre core before it explodes again into blue light and forms the nebula explosion. Proprietary fluid dynamics solver Squirt played a crucial part in the sequence. It gave effects artists the ability to have the simulation contract using negative expansion, where the density gets taken out of the simulation.

The explosion was rendered using in house renderer DNB. A custom shader Inferno made it possible to use the fluid simulation as a light source to light the surrounding clouds and smoke. Says Cook, "Because of how creative the brief was, a lot of it ended up in the 2D realm, with matte painting, etc. The team had to experiment with the way that it moved and the timing of it, involving more work on top of that to integrate all the elements, because it was constantly evolving." Squirt was also used to create simulations for ground-based effects such as exploding fountains, dust elements, swirling paper and debris.

Adds Bickerton, "As a final texture, we photographed laser textures created by firing a laser through a clear plastic skull. These animating ribbon textures were added to explosion imagery giving a random sub-atomic texture."

Digital Churches Interiors
Also working on Rome environments was CIS Vancouver. Led by VFX Supervisor Mark Breakspear [2], the team created three CG church environments: Santa Maria della Vittoria, Santa Maria del Popolo and St Peter's Basilica. All three churches were off the shooting grid, so visual effects were the only way they could appear in the film.

The approach was to recreate them digitally, stitching together thousands of digital photographs taken at the real locations, and projecting them onto accurate computer models of the buildings. "We used three Canon EOS 1DS MK2 cameras, with gave fantastic quality in the low light levels of the churches" Breakspear says. "We were not able to get HDR in the truest sense, but we managed to get around that by also shooting with a stills film camera. We used Kodak 5218, which was the same film stock from the main unit shoot. By taking image on film, we were able to make use of the wider range from those images for windows, highlights and areas where we wanted to add detail into the shadow areas. We scanned our film images at 5040 × 3684 @ 240 dpi, while our digital images were captured at 4064 x 2074 @ 72 dpi."

For Santa Maria della Vittoria and Santa Maria del Popolo, the art department built one set that could work for both churches, with a little redress. For St. Peter's Basilica, the actors were shot on an entirely greenscreen set, with only the floor and a partial column to orient the vfx team. "We ended up replacing the column and the floor, but don't tell anyone," Breakspear chuckles.

"For Popolo, there were lots of issues with the practical set not lining up with the real location photography, due to creative license being heavily used in its construction. In the end, we were able to bring both locations together by duplicating areas where needed, and hiding other areas that no longer existed in the practical set. Santa Maria del Popolo was the only church we built using LightWave [12], everything else was built in Maya. When we worked on The Da Vinci Code, we did all our shots in LightWave, as at the time, our pipeline was more centered around this package for this type of work. Four years later, our pipeline has moved toward Maya [13], and although we still love what LightWave can do, it made sense to split the builds across both packages. All the churches created in Maya were rendered in mental ray [14]. Each frame was generally broken out into 25-40 specific layers, giving the compositor great control on adjusting the look we wanted."

For Santa Maria della Vittoria, Bickerton shot a miniature of the ceiling and the resulting images were digitally projected onto a CG roof that could be placed in any of the shots. Fire, embers, heat haze and smoke enhancements were also added to the practical fire. All of the compositing was done in Shake.

For St Peter's Basilica, the task was far greater, for CIS had to create just about the entire environment, other than the actor. "Normally, you would try to avoid this, as that much CG real estate is typically very hard to pull off convincingly," Breakspears observes. "Our CG Supervisor, Karen Ansel felt that we could master all the nuances required to fool the eye, and together with 2D Supervisor, Martyn Culpitt, worked out all the required layers and approaches needed to pull it off. The end result is pretty amazing!"

Simulating Hypoxia
Another landmark location in Rome, the Pantheon interior was built as a partial set that was extended digitally, both vertically and horizontally. VFX Supervisor Richard Higham and his team at The Senate built a hybrid of the rest of the location using photo reference of the set and the real church. They built the domed ceiling and oculus, and created a fully CG shot for Langdon's POV as he first walks into the building.

In a dramatic sequence, Langdon and a Swiss guard find themselves entombed in a vault and are slowly starved of oxygen. Referring to their personal experiences with NASA astronaut training when shooting Apollo 13, Howard and Todd Hallowell were keen to depict the visual degradation caused by hypoxia (oxygen starvation). "The Senate tested various looks based on Todd's vivid descriptions," Bickerton says. "At the same time as we tested the RED camera for potential plate usage, we shot material for the tests whereby the camera was hand-held tracked down corridors. The footage was stabilized in X and Y, but also in Z to give a digital contra-zoom effect. On top of this, vignetting mimicked the tunnel vision side effect of hypoxia, along with trailing highlights around the periphery of the image."

Anti-Matter Issues
MPC (The Moving Picture Co.) also did significant work on Rome's environments, but the team mainly focused on the design and conception of the antimatter and its journey through CERN's Large Hadron Collider.

Bickerton says that Howard thought it was important for the audience to get a sense of how anti-matter was created and gathered. "To start the process, I assembled RED footage that we shot at CERN (European Organization for Nuclear Research) in Geneva, together with pieces of CERN's own animations to make a crude sequence. From that, MPC's layout team animated a 40-second shot that started in the CERN control room (a fictitious invention designed for the film) and passed through its glass partitions into the ATLAS detector. At this point, the view became 3D X-Ray as the camera traveled through the multiple layers of the Detector to its heart to follow the complete process."

Liaising closely with Bickerton, MPC VFX supervisor Richard Stammers oversaw the London team with the help of Kevin Hahn as CG supervisor and Richard Baker as 2D Supervisor. The interior of the facility, including the outer parts of the Atlas, were built based on photographs, using image modeling and camera projection techniques. To re-create the X-Ray look of the interior of the Atlas detector, MPC was provided with a complex CAD model from CERN, and Kevin Hahn adapted this, so he could develop the look in RenderMan. "For sections of internal pipes for which we had no access to, we used reference photos sourced from the Internet to construct CG sections to join the multiple environments," Stammers explains. "Particle collisions and anti-matter FX were created using Maya, and combined using Shake."

Adds Bicketon, "In order to visually connect the anti-protons to the final explosion, the laser textures were again used as trailing energy emitting from the particles."

The majority of the rest of MPC's work revolved around large-scale CG environments and set extensions to locations in Rome and The Vatican City that were impractical or forbidden to shoot in. The Piazza Navona and Passetto (a 40-foot high walkway linking the Vatican to Castel Sant'Angelo) sequence both required digital set extension to partially built environments. An 80-foot long reproduction of half of the Piazza Navona was built on a parking lot in Los Angeles, the opposite half was a greenscreen set which was later replicated using camera projections taken from the real piazza. For the chase at the Passetto, MPC again extended a greenscreen stage using real environment projections and day for night reference photography.

The team also produced the title sequence, including logos, a fully CG Papal ring in glorious macro close-up and handled the 2K dailies for the production.

Invisible VFX
Bickerton notes that, in terms of story, this film was less beholden to the original material. "For Da Vinci Code, Ron had strived to be faithful to a book that had been read by so many. But for Angels & Demons, with Dan Brown in complete agreement, he set out to make an adaptation that was much more of a race against the clock thriller. Yet, the visual effects mandate was the same: to be invisible in our enhancement and creation of backgrounds -- with the addition of providing a believable spectacle when the antimatter bomb explodes. I hope we have."

Alain Bielik is the founder and editor of renowned effects magazine S.F.X, published in France since 1991. He also contributes to various French publications, both print and online, and occasionally to Cinefex. In 2004, he organized a major special effects exhibition at the Musée International de la Miniature in Lyon, France.