'The Nativity Story': Making the VFX Look Real

Sometimes when the stars align, miracles can happen. Such is the tale behind the making of The Nativity Story (opening Dec. 1 through New Line), based on the story of Mary and Joseph and the birth of the baby Jesus.

Director Catherine Hardwicke's main focus was to remain as historically accurate as possible, with attention to detail on the architecture, the customs and what information was available. There is a theory that the Star of Bethlehem was actually the planetary convergence of Mars, Venus, and Jupiter, an event recorded by the Chinese recorded around 5 B.C. The Nativity Story used this and the struggle of Mary and Joseph as a basis for the film.

Filming

Filming took place in Morocco and Matera, Italy, the same location used for . Before shooting began, Theresa Ellis Rygiel was brought in to supervise the visual effects. That was March 8, and within four days, Rygiel was on a plane to Italy to stay for eight weeks of filming. "I think a lot of people were wondering why the vfx supervisor was showing up before the second [assistant director], but it was to Catherine's advantage in the end," Rygiel explained. "She had me as her digital art department for about two months prior to post-production."

Since most of the film was shot outdoors, Rygiel and cinematographer Elliot Davis tramped through the Italian hillsides, discussing the lighting, times of day, day-for-night, dusk shots, campfires and lighting hillsides with giant rigs. Rygiel started with precomps of the show, putting together stills of the locations with different skies, planning out matte paintings and composites of each of the effects prior to the shoot.

To assist Hardwicke with visualizing, Rygiel contacted Manfred Buettner of TVT-Postproduction in Berlin, who sent Carsten Woithe and Florian Gellinger to Italy. There they set up a 3D previs department, using location shots and buildings created at Studio C in Guatemala. Rygiel also selected Hammerhead in Studio City, California, to handle the original estimate of 60 digital effects shots.

Hardwicke made the decision to minimize the use of greenscreens, preferring to see people reacting in natural environments. In some cases, the sweeping reveal shots couldn't accommodate greenscreens, and in others they would have blocked the environment. Since a steadicam was used for filming, there was no motion control to do a second pass, so Rygiel was anticipating a lot of roto work.

The decision was made to avoid the day-for-night because it lacked reactive lighting from sources such as campfires and candlelight. "We ended up trying to get neutral plates with correct skin tones, bringing the blacks and whites into range, and shooting things as close to dusk as possible," said Rygiel. Her most challenging sequence was entering Bethlehem at dusk, with a long steadicam track following Mary in labor and Joseph frantically looking for a place to stay. "We wanted to replace the sky with something moody, with the sun setting to portray urgency. We started out with a bit of horizon lighting and tried to bring it down in post. We had the flexibility to crank it down even further because we had all the information in the frame, in the darks as well as the lights."

The Shot List Grows

The vfx were originally scheduled to be done entirely at Hammerhead, but shortly after the rough cut was viewed, the decision was made to increase the scope of the film. The post schedule remained at only two months, with several screening deadlines and a vfx shot list that was expanding from 60 to roughly 200 shots. To meet the demands of the deadline, in mid-August New Line brought in Jamie Price as vfx supervisor and Stephanie Allen as vfx producer. Rygiel continued to work with Hammerhead, which handled about half of the shots, while additional vendors Frantic Films, Digital Dream, Digital Domain and yU + co shared the remainder.

Price and Allen's primary task was to get things organized. They began by breaking shots down, grouping them together, making sure everything was appropriately awarded. Allen worked with vfx editor Paul Wagner, setting up a consistent shot naming convention and a database, while Price met with Hardwicke to understand her vision to communicate to the venders.

Multiple Shots, Multiple Venders

Digital Dream was one of those vendors, taking 31 shots of mostly set extensions or compositing with 3D tracking. A completely MAC-based company, Digital Dream used PFTrack, Shake, After Effects, Maya and Combustion, and worked on the project for about 10 weeks with a team of 12, finishing up mid-October. Jerry Pooler was Digital Dream's vfx supervisor and Corinne Pooler was the vfx coordinator.

The longest shot was in Nazareth, with up to three people dedicated over six weeks, some tracking, some painting, some compositing and all working simultaneously. The most challenging was the tracking. They had to roto out the background, add architecture, lights and flickering from fires in the windows, campfires, trees, wheat fields and various activity such as people or wandering sheep, corner pin buildings and adjust the models. Everything was 3D tracking with matte painting extensions, and each item had to be tracked differently to accommodate the perspective.

There is a scene where Mary and Joseph are crossing a rushing river. On set there were some jet skis churning up the water near the camera, but beyond the camera the water was placid where the river needed to look threatening. The Poolers were leaving for a trip to the Colorado River, so they shot video while there. The water plates had to flow in the right direction, and around the rocks. Jerry Pooler took the water plates, split them in, color matched and added shadows.

"We basically fixed a lot of things," said Jerry Pooler, who brought 37 years experience to the project. "They wanted things like the textures on walls changed. It wasn't that it was a challenging show; it was more that the time frame on it was so short, the pipeline had to flow well."

Digital Domain took on a few key shots, with Kelly Port as the fx supervisor. When the angel Gabriel appeared, he was shot as a human-like presence, but it was later decided to give him more of an ethereal quality, so Digital Domain added a transparency around his edges and a glow around his head. Since they didn't have any clean plates and all of the shots were moving cameras, they had to do some extensive plate reconstruction and tracking. The film is bookended with the Massacre of the Innocents, a scene shot at night. There was no exposure in the sky, but there were some practical lighting effects. Digital Domain replaced the sky, adding clouds and lightening, which required some tricky black on black roto and tracking with little camera information or tracking markers.

Another rapid turnaround shot that utilized several different techniques by D2 was the last of the film, with the flight to Egypt. The shot is a crane up over a ridge to reveal Joseph and Mary in the midground and the pyramids and Nile in the distance. "That was a concept that came out of the cutting room in post," said Price. "D2 used a foreground element craning up over a ridge, which was rotoscoped and tracked, a midground element with Joseph and Mary projected onto 3D geometry, and a 3D matte painting."

Goop and Light Beams

A live-action newborn was used in the birth scene, and for time and safety reasons, the baby was kept clean. "We wound up with not only an immaculate conception but also an immaculate birth!" said Price. Frantic Films was charged with making the baby "goopier." To accomplish this, they did an extensive 3D tracking, matchmoving a baby model with a reflective wet texture for compositing. Each fold on the baby, and each movement were tracked so that the reflective elements would stick, and Frantic used multiple layers to easily adjust the amount of "goop."

Hammerhead, who handled the bulk of the work including the star beam, sky replacements, and all the Jerusalem and Bethlehem matte paintings, also worked on the grotto interiors for the birth scene.

The star's beam was to shine through a hole in the ceiling and illuminate the infant, but the smoke machine couldn't be run with the live-action baby, so Michelle Eisenreisch, vfx producer at Hammerhead, had had her team add the beam to about 30 shots. "This required rotoing Mary and Joseph, working with dark fluffy hair over a dark background, creating a whole list of technical challenges. We had a good roto team and compositor, and Rebecca Marie as the roto supervisor."

100 Shots in Two Months

In the Bethlehem dusk scene where Mary and Joseph seek shelter, the only buildings were in the foreground, so Hammerhead extended the city and did a sky replacement, working with footage shot with steadicam and no greenscreens. There was extracting in the foreground including detail such as palm trees, and tracking in the background. The foreground was in shadow from a hill that was off camera while the background was in daylight, requiring darkening the hill in back. The Bethlehem shots were some of the first to be scanned and the last to be finished, taking three compositors two months to complete.

There were only one or two shots with greenscreens, so all the sky replacements and set extensions and the matte paintings, which were the majority of the work, were roto and tracking intensive.

Eisenreisch had about 30 people on the Hammerhead team, plus Carlos Arguello and his company Studio C in Guatemala. "We had Carlos and several artists in-house, plus a bunch of artists back in Guatemala that were able to work during production, providing models and previs elements to Theresa while she was in Italy shooting. They would also turn those around into matte paintings and ftp those to us." Studio C would break the paintings into layers to hand over to the Hammerhead comp department.

Hammerhead relied on their own roto software, ras_roto, and partner Thad Beier's 1999 Academy Technical Achievement Award-winning tracking software, ras_track, plus Shake for compositing. Todd Perry and Beier developed matte painting integration into the compositing phase. The final paintings would be delivered in layered Photoshop files. Perry would take those layers, apply the distances derived from the 3D track, place the layers in 3D space and render out each layer as a new element with the proper track applied. These new layers were then turned over to the compositors. This way the compositors didn't have to place the layers; they came in a way that was already tracked distance-wise to show the proper parallax.

Keeping It Real

Hardwicke's goal was to treat this as a realistic depiction of the event, so there were a number of shots showing the planets converging over time. "It's difficult to photograph stars using a motion picture camera with the exposures needed for film, so sky replacements were used to keep the stars consistent while drawing attention to the gradual convergence," Price explained. "Stars are always a technical challenge because when they get down to one pixel it's unpredictable what they are going to look like on the screen." What made it more difficult was that he had star shots spread across three vendors, and had to make sure stars were consistent and twinkling the right way.

Under an extremely short deadline and multiple locations, the teams pulled together to create some seamless work. "Jamie's creativity and inventiveness was key to getting the level of artistry we did from the crews at each effects house," said Allen.

Renee Dunlop has worked in film, games and multimedia since 1993. She currently works at Sony Pictures in Culver City, California, and freelances as a Maya lighting digital artist and as a writer for several trade publications.