A Golden Opportunity for 'Hellboy II'

Dozens of new creatures, hundreds of character animation shots, some of the weirdest characters ever put on screen... Hellboy II: The Golden Army (opening today from Universal) definitely plays in a much higher league than 2004's Hellboy. The sequel has Hellboy (Ron Perlman) and his team of misfits confronting a horde of mythical creatures who rebel against humanity.

The movie was the largest character animation project ever undertaken by lead vfx vendor Double Negative. The facility had just delivered Grawp for Harry Potter & the Order of the Phoenix, as well as the saber toothed tiger and the terror birds for 10,000 BC, so lessons learnt from these prior projects were incorporated into the nascent creature pipeline.

"The real challenge here was to gather all the different pieces of a creature pipeline developed on other shows," says co-CG Supervisor Andrew Chapman. "We had to form a complete character pipeline that would handle the vast array of different creatures we were animating, as well as a very high volume of creatures within a single scene, up to about 5,000 Tooth Fairies in some shots."

Double Negative's pipeline was modeled in Maya, ZBrush and XSI; rigging and lighting were done in Maya; effects work were done in both Maya and Houdini; rendering was done in RenderMan through a proprietary Maya-RenderMan tool called Rex; and compositing was done in Shake.

Animators used Maya as the primary animation software. NaturalMotion endorphin was used a little in the Golden Army sequences. "We developed some new tools to help with shots involving large numbers of characters, working on the principle of caching," explains Animation Supervisor Eamonn Butler. "The animator would select a rigged character and cache it out to geometry. It could be switched back to an animatable rig at any point, allowing the animators to have a large number of characters in the shot at any one time without suffering too much slowdown. This was particularly useful for the Tooth Fairies sequences."

Splitting Up the Workload

The modeling and rigging teams started work as soon as the project was awarded in February 2007. "By starting immediately, we were able to keep the crew smaller and more manageable, so more focused and highly skilled in their crafts," Senior Visual Effects Producer Steve Garrard notes. "Due to the long length of the shoot, it was agreed to turnover certain sequences whilst the film was still shooting principle photography. Although feedback wasn't as instantaneous as it would become in post, this was key as we ended up providing CG animation for over 500 shots.

"Because of the size and scale of the project from the outset and intrinsic to our plan was Universal and Guillermo allowing us to sub-contract a certain amount of the work. We assigned shots to Cube Effects in Hungary, and to The Senate, LipSync and Baseblack in London. This enabled us to stretch the incredibly tight budget as much as possible, and still maintain the high quality demanded by Guillermo [del Toro]. Also, we took the decision to split the job internally into two parts. Both team had their own production and supervision teams, whilst still being part of the team led by myself, Eamonn Butler and overall VFX Supervisor Mike Wassel."

The first team was led by Digital Effects Supervisor Andrew Chapman, VFX Producer Andy Taylor and VFX Coordinator Antonella Ferrari -- and the second one by Digital Effects Supervisors Adrian De Wet & Justin Martin, VFX Producer Moriah Sparks and VFX Coordinator Karen Clarke.

Design and Rigging Challenges

Most of the creatures were built based on scale maquettes designed at Spectral Motion and Solution studios. However, the Golden Army soldiers started off as a rough drawing and were developed at Double Negative by Graham Jack and Jay Davis and their Golden Army crew. "During the built process, we changed the gait of the soldiers walk cycles to enable the Golden soldiers to feel heavier," Garrard remarks. "So, the legs changed length and this drove how big the practical steps were built on the Golden Army chamber. The other major evolution that we did to the creatures was the addition of the 'pimp coat' onto the Elemental, the giant monster that appears during the climax. This jacket of vines helped give it scale."

The members of the Golden Army turned out to be the most complex build and rig, due to the sheer amount of parts, and their complex and intricate nature. The team describes them as beautiful Swiss timepieces that had to emote anger, vengeance and pathos... The robots also proved difficult to animate. "Every joint, plate, cog, etc. needed to be hand animated, and there were hundreds of moving parts," Butler says. "Every time a robot took a step, each plate of armor, rib, joint, etc., needed to jiggle at a particular frequency to indicate the weight hit. Also, the robots have the ability to rebuild themselves. All of the robot parts that litter the floor had to be rigged and animated, as the robots are progressively destroyed and later rebuild themselves..."

Jack was CG lead on the sequence: "We had a main rig controlling the performance, but there were also secondary cogs inside the creature that were primarily driven by its movements. On the hero moments, when the creatures were reforming, the rigs were all custom rigs and hand-animated."

Double Negative's asset management system was used to control the level of detail of the robots and their visibility in the scenes. The team tried to keep the geometry as light as possible, but employed ReadArchives to pull the high detail geometry at render time.

Multiple robots shots were rendered by breaking down the scenes down into foreground, midground and background. "We also ran a preliminary bake pass before rendering the beauty pass," Martin explains. "It helped speed up subsequent re-renders for lighting changes. The beauty passes had a set of secondary output passes to give the compositors more control over the look of the robots, which helped reduce the re-renders needed. For a few of the wide shots, we used particle based instancing of hand-animated cycles."

Of Stones and Swarms

Among the movie's many unusual characters, the stone creature was one of the most challenging to visualize. Its very nature implied a tricky combination of hand animation and simulation. "Our goal was to achieve a complete sense of performance driven by keyframe animation enhanced with dynamics," says Andrew Whitehurst, CG lead on the sequence. "To that end, we did the key frame animation on a non-dynamic giant first to get the basic performance. In order to add the rocks breaking up in a controllable way, we developed a system that would allow us to make certain chunks of rock become dynamic at defined moments. For example, if the giant slammed his hand down, we could choose some rocks around that arm to become dynamic when that event occurred. It enabled us to direct when the dynamic moments happened in a way that emphasized the keyframe animation, rather than trying to simulate everything at once.

"We engineered the system so that we were able to fine-tune the simulation on any particular boulder. We could make all the other rocks in the simulation non-dynamic again, and just tweak the simulation for that one boulder until we got what we wanted. It was a case of using simulations to get something naturalistic, but controlling when and how the dynamics happened to make them work aesthetically and narratively."

Due to their sheer number, the tiny Tooth Fairies presented their own set of unique challenges. The hero creatures were created using hand animation to obtain a defined performance. The rest of the swarm was animated via a particle-based system that used cycles of hand-animated behaviors. "The animators would keyframe a number of guide creatures for the technical directors to follow," Butler explains. "The TDs would then implement Double Negative's proprietary swarm tool to create a particle simulation. This sim would form the basis of the speed and direction of the creatures. Animation cycles and behaviors were sampled and combined to create the creature's actual motion. Many iterations were needed at the start of production to find the behaviors that worked best."

A Mammoth Effort

During the climax, the colossal Elemental makes a spectacular arrival through a city street, ripping pavement apart and tossing cars around. The destruction effects were created using dynamics, with cloth simulations added to crumple the cars panels. Additional dirt and debris passes were rendered using Double Negative's fluid simulation and volume rendering pipeline.

The Elemental itself was particularly challenging from an animation standpoint. "It was mostly tentacles with no jointed limbs," observes Colin McEvoy, animation lead on the sequence. "It took a lot of work to animate appealing shapes and motion with its many tentacles because of the complexity of the animation controls. It was also extremely large, but needed to move dynamically."

Adds Butler, "The process of animating the Elemental was centered around the primary animation principles of line of action, silhouette and negative space. As the size of the creature meant its entire form was not in the frame for most of the sequence, we tried to build appeal in the animation of the creature by focusing on the abstract shapes created in the composition by the flailing tentacles. Once we had established the basic poses, we would build on them by animating passes to create weight and dynamism."

The vegetation on the creature was a secondary animation system that was run as part of the lighting pipeline. Other effects passes, including steam and drips, added extra layers of detail and realism. "There was a huge amount of work in compositing putting all this together, bringing in other elements of steam and atmospherics, pyrotechnics, etc., to make it all work," Martin remarks.

Digital Enhancements

Parallel to this full CG effort, the team was dealing with humanoid creatures that "only" required digital enhancements on practical make-up effects. It included creating Abe Sapien's eye blinks in 2D, a task assigned to Budapest's Cube.

The character that needed the most digital enhancements was Wink, a huge creature built on steroids. "Wink often needed to have his mace fist replaced digitally for his fight scenes with Hellboy," Butler observes. "We also provided a completely digital Wink for shots in the Troll market where the actor couldn't safely perform the stunt or action required. In one particular shot in the sewer, we are introduced to Wink as he emerges from the shadows. The director shot it practically but didn't like the lighting when he saw it on film. He wanted to keep the character in shadow and reveal him later in the movie. We ended up tracking the practical Wink and replacing him entirely with a differently lit Wink!"

Adds Martin: "The Troll Market sequence is a good one for the combination of practical and digital effects. It's predominantly practical, but there are a lot of digital additions -- from something as simple as adding digital eye-blinks, to replacing part of a main character (like the mouth and jaw, or a mechanical fist) to putting full digital characters into a shot among practical creatures (guys in suits)."

Another key digital enhancement was the addition of CG flames on Liz Sherman, a girl who has the ability to control fire. First, actress Selma Blair was body-tracked and geometry was used to drive a fluid fire simulation. Eugenie Von Tunzelmann was the CG lead on the character: "The effects TD would tweak the simulation on a per-shot basis. The fire had to react appropriately to Selma's gestures and her performance, such as any emphasized words. We added fields, collision objects and so on if necessary. Both the fire simulation and the renders were achieved using proprietary software."

Part of Double Negative's involvement also implied creating and animation highly realistic digital doubles for Hellboy, Abe and Johann. Even though Hellboy only had five full CG double shots, the team had to contend with many CG fists and CG tails. The characters were first cyberscanned on set by PCA, and then extensive digital stills were taken by the Double Negative crew. The team also shot reference footage on HD of the characters doing a range of poses and actions. The characters were then all hand-animated. No motion capture was used. "Hellboy flying out of the window at the end of the Tooth Fairies sequence was all-CG apart from the 'Casino' shot," Garrard says. "Abe had one full-CG moment swimming in his tank, and Johann was a CG double when inflating and deflating."

Creativity and Enthusiasm

Hellboy II ended up being one of the toughest shows ever undertaken by Double Negative. The variety and volume of characters and creatures meant that the team was busy from the very beginning. "Conceptually, the movie gave us a lot of creative latitude and it proved to be a very rewarding and creative experience," Butler concludes. "Guillermo is a visionary director and he inspires everyone around him. It is hard not to be affected by his enthusiasm!"

Alain Bielik is the founder and editor of renowned effects magazine S.F.X., published in France since 1991. He also contributes to various French publications, both print and online, and occasionally to Cinefex. In 2004, he organized a major special effects exhibition at the Musée International de la Miniature in Lyon, France.