Greenscreens and editing tricks didn't cut it for Blades of Glory, so Thomas J. McLean finds out how Rainmaker's new facial performance capture took the gold for this ice skating comedy.
Will Ferrell may not know a Lutz from a flutz, and Jon Heder is unlikely to pull off a perfect salchow in competition. But Rainmaker's facial performance capture techniques make it possible for both actors to look like they're carving up the ice for real in the new DreamWorks/Paramount comedy Blades of Glory (which opened March 30).
Key to film's success was making sure the audience believes at all times that Ferrell and Heder were on the ice executing moves that take real skaters years to learn, let alone master. Greenscreens and editing tricks wouldn't cut it, leading directors Josh Gordon and Will Speck to take on Mark Breakspear of Rainmaker as their visual effects supervisor.
"We wanted to show Will Ferrell skating in the frame, full frame, skating all the way around and then doing a jump and landing and see him act and see him be Will Ferrell," says Breakspear. Rainmaker started off with a test to see if their techniques would work, and ended up developing a proprietary system that used facial performance capture to create 3D digital faces that could be placed on the body of the doubles who did the actual skating, often replacing the entire head. "That way, Will Ferrell could skate in any angle in any scene," he says.
Not Cheap, But Affordable
Breakspear says perhaps the most significant aspect of the process is that it's inexpensive enough for "90%" of films to afford. "This kind of process has really only been available to people making movies with really limitless budgets, because the technology has always been so extreme and cutting edge and new," he says. "I'm not saying it's cheap, I'm saying it's out of the stratosphere and into everyone's potential."
The process used high-resolution motion capture to record actors' facial expressions to create accurate faces to replace those of the doubles who perform the actual skating. (Skater Chad Brennan doubled for Ferrell, and Patrick Hancock for Heder.) Breakspear suggests the directors knew immediately that this was the right way to go even though the process was untested.
"I think it's the performance capture that makes people a little nervous," Breakspear adds. "You're putting a lot of trust in the visual effects crew to make sure that the joke doesn't get lost in the process."
The process went through six months of R&D, during which the process completely evolved twice before it was finalized. The first step was to create very accurate models of the actors' faces. They used plaster cast molds to create clay models of the actors' faces. They took them to XYZ RGB, which scanned the models at a resolution of 50 million polygons per face.
Anatomy Lessons
Rainmaker had the actors make crazy expressions and analyzed in detail how their faces move. The work involved figuring out how each actor's muscles work under the skin and setting limits to how far they could move in any one direction. From this, they determined between 120 and 150 key points that must be recorded to accurately capture the performance, centered around the eyes, cheeks, neck and forehead. With the points set, reference dots were added to each actor's face in exactly the same spots.
Taking an early edit of the movie, they put the actors into their makeup and added the dots. They were then seated in a rig that used mirrors and three HD cameras and one film camera set at specific angles to capture the data from every point. The actors then did their performances, doing multiple takes with the director so they had as many choices later on as to which to use. Breakspear says the cameras recorded both the texture of their faces, which were lighted very flatly, as well as the position of all the reference points.
A Breakthrough Moment
Using MatchMover Pro, Rainmaker's crew analyzed the data and converted it into a point cloud. Applying the cloud data to the digital models required a lot of custom coding, but in the end created highly accurate results.
"It's like one of those moments of 'Eureka!'" he says. "You suddenly have a model of Will Ferrell that suddenly comes to life and every nuance of his face that you see in the real person is transferred across to the CG model," Breakspear continues. "That's one of the things that's been the holy grail of CG faces."
Rainmaker used HDRI data captured on the set to exactly match the lighting, and added skin texture and subsurface scattering to the models. The one thing the technology still cannot capture is eye movement, which had to be done separately using digital photography and filmed footage as reference.
For compositing, Rainmaker used one team to rotoscope out the doubles' faces and another to do the tracking. To make things easier, the doubles wore wigs that matched Heder and Ferrell's hair so that in almost every shot only the face and neck had to be replaced.
Through the Pipeline
Completing the shots began with giving the performance capture footage to editorial and asking them to choose expressions for each shot. Simple faces were tested and approved, and running the data through the pipe took anywhere from a week to three weeks, then went off to lighting and was finessed until finished. The average shoot took on average a month to complete.
Breakspear suggests it was a battle to convince people that this process was preferable to simpler 2D effects, but directors Speck and Gordon immediately knew this was the way to go to ensure the final result had the grace and speed of real figure skating.
About half of Rainmaker's 300 effects shots involve face replacement and about half of those have two faces per shot. Other effects included altering the background stadium so the L.A. Sports Arena would look like three different facilities in different cities, complete with crowds. Rainmaker was the overall supplier for the film, with about 80 shots sent out to other houses. Digital Dimension and Image Engine did some comp shots and Soho contributed renderings of the crowds using Rainmaker's Massive setup.
The crew totaled about 150 people, who did five months of post-production work from October through February, Breakspear says. That includes about 25 rotoscopers, 25 trackers, 15 tracking people, 30 compositors, 30 CG people and a management team of about 20 and a team working with editorial.
Breakspear says these tools gave the filmmakers an enormous amount of flexibility to constantly refine, alter and splice together specific elements as they searched for the funniest moments. "We had opened a Pandora's box and gave the filmmakers the ability to make some pretty cool decisions along the way that otherwise have to be made on set, that lock you into a certain performance, and then you're done," he says. "They have the opportunity to change things that aren't laughed at enough. It gives the filmmakers a new set of tools."
Thomas J. McLean is a freelance journalist whose articles have appeared in Variety, Below the Line, Animation Magazine and Publishers Weekly. He writes a comic book blog for Variety.com called Bags and Boards, and is the author of Mutant Cinema: The X-Men Trilogy from Comics to Screen, forthcoming from Sequart.com Books.
Tom McLean has been writing for years about animation from a secret base in Los Angeles.
