<em>Monster House</em> Sneak Peek: Q&A with Jay Redd

[1]

Jay Redd heads up the visual effects team on Monster House. [1]

We talked with visual effects supervisor Jay Redd about the evolution of performance capture on Monster House from exec producers Robert Zemeckis and Steven Spielberg. It’s the next all-CGI film after The Polar Express [2] to use Imagemotion performance capture technology from Sony Pictures Imageworks.

Barbara Robertson: Did you use performance capture for all the characters in the film?

Jay Redd: Yes and no. We have a great cast and we’re proud of our performances on this film — there are wonderful moments from each character. But sometimes when we get into editing, we might find something would work better if it’s keyframed. What I’m excited about is that there is always a mix.

Also, this is primarily a motion capture show, but a number of characters were keyframed. There are some characters that embody human ideals but are not built like humans. For example, the house is a prominent character, a very complex, emotional creature, but it wasn’t built like a human, so performance capture wouldn’t give us what we wanted. But, because it’s an emotional character, we had an actor give us 3D reference. You get a quick glimpse in the trailer. I don’t want to spoil the film… there’s much bigger stuff that happens.

BR: Did you use the same actors for the performance and the dialog?

JR: It’s all identical. We captured voice, body and facial performances all at the same time.

BR: At the same time?

JR: Imagine yourself at a small theater on the round. We could have five or six actors on set performing together for minutes at a time. The actors loved it. Also, we’re shooting a virtual film, so Gil [Kenan, the director] could be on the set working with the actors, helping them get the most dynamic performance.

BR: It sounds like an improvement over the performance capture for Polar Express. What’s different?

JR: Technology is constantly changing; every film is a new generation. Computers get faster, cameras have higher resolution. Polar was groundbreaking because we could capture face and body, but we had to do some of it separately. We took it up a level on Monster House.

BR: In what way?

JR: We could capture more actors in the same place and our actors are playing themselves. It’s all about the stage. I worked with [motion capture supervisor] Demian Gordon, who worked on Polar and on Matrix [3] as well. I told him I wanted to have a minimum of five actors on stage. Our motion capture stage . . . imagine a 20 x 20 foot cube 16 feet high. That’s the volume we could capture people in. On Polar, we had two 10 x 10 capture spaces. On Monster, with the 20 x 20 space we could get more detail from all the performances. We had 200 infrared cameras lined up so they could see reflected light from the markers on the actors. We needed so many cameras to see the right angles on all the actors.

BR: How long can a shot be?

JR: The more actors in the volume, the less time you can shoot because you’re feeding more data through the system — the cameras are seeing more dots. When we had one or two actors, we could shoot for several minutes. When we had six people, we might be limited to three or four minutes. Depending on the actors, we could sometimes shoot for six minutes — an entire scene, five takes. It was like watching theater happen.

BR: How many dots did the actors wear?

JR: It depended on the actor. We usually had between 60 and 80 dots on the body depending on what we needed to capture. The faces had around the same depending on which muscles moved. We did face casts of each actor and a range of emotion tests and created a set of dots specific for each actor. The facial system at Imageworks also caters to each performer. We worked with the makeup artists to create a template for precise and consistent placement of the markers every day.

BR: Did you design the characters based on the actors?

JR: Our characters are stylized — we’re not matching real people. They’re bipeds, but their proportions are different; they’re cartoon characters. Some of the characters are dramatically different from their live-action counterparts, but some of the best performance captures we have are from actors who somewhat physically match their digital counterparts.

[4]

The first released still of the protagonist from Monster House. [4]

BR: Were the performance capture sessions like filming actors on a set?

JR: Even though we did the camera moves later, we still had to think about how we would shoot the scene later and stage the actors appropriately. If everyone was blocked up together on the stage, we wouldn’t be able to get a camera in later. So we had to talk about how we were going to shoot the scene when we were on the motion capture set. Gil brought on a dp, Xavier Pérez Grobet, to work with us to help stage the actors and set the cameras appropriately. We had six videocameras. We put them around the stage so that they’d roughly mimic the way we’d shoot the film later — on one side or the other, for example.

BR: What happened after you captured the performances?

JR: We selected the performances based on the video and that’s what editorial used for a rough cut. They created a really, really rough cut, almost like a storyboard. Then the selected performances were collected and roughly processed and we created a rough layout the director used to create rough camera moves. At this point, data from the captured bodies had been put onto the digital counterparts, but not the facial performance. The hands didn’t move — it was pure body language.

Editorial worked out the timing from the rough cut, the facial performances were selected, and Imageworks did what we call final integration. They refined the motion capture data, added the facial performance data, and gave it to the animation department for final touches. And then it went into what we call Wheels for final camera moves.

BR: Final camera moves?

JR: We wanted our whole process from beginning to end to feel handmade. We didn’t want it to feel sterile; we wanted quirks throughout the film. Wheels is a virtual camera setup, but the camera operator and director use familiar tools. We have a room with a 50-inch plasma screen and a grid that reads sensors on a camera mount, a steadicam and so forth. The director brings in a traditional camera operator who uses traditional tools to move the camera in the virtual setting. They can change the point of view [in the virtual environment] in realtime: We can put a shoulder mount sensor on the camera so as the operator moves, the view on the plasma screen changes. We can give a hand held feel to a camera move that was a dolly shot in the rough layout.

Also, in the rough layout we don’t know where the character is looking. Wheels comes after animation. We know where the eyes are, the position of the bodies. It’s almost like having the actors on set again. The director can do 100 takes of the actors without having them get tired, without their makeup running. It’s a blessing and a curse. You can play, experiment, explore, but at some point, you have to make a decision.

BR: Do you like this way of making a film?

JR: I’m very excited about this. Some people are threatened by performance capture and I was one of those people. But, it’s a new way of working; it gives you immediate 3D reference. For us and the director it gives reference and immediate feedback to 3D world.

When we first got together, Gil brought conceptual artwork. We had many meetings talking about the style of the film. Gil said, “Let’s do something different.” We both love stop motion and edgier cinema, so we decided to put those elements together with motion capture and keyframe to create a brand new look.

People ask, “Why not keyframe?” I don’t think that’s the question. The question is, “Why wouldn’t you do performance?” What we’re doing is an elaboration on puppetry. Gil and I refer to the characters as puppets, the most articulated stop motion puppets in the world. Instead of a human performing a puppet by hand or moving a marionette with strings, our puppets are performed by actors, and that performance is interpreted through human hands. The performance is taken into computers where people sometimes animate on top and sometimes replace to end up with something new. [Our puppets are] so fluid, so magical, you’re taken in by them.

I think it’s a new way to make films; a specific way to make films. It’s a new genre.

BR: Are you finished?

JR: We’re in post, but… it’s actually pretty funny. Post on a film like this is like making a movie. We captured performances for 40 days; that was our shoot schedule. We have the animation and camera. Now, we’re making a CG feature. Post is putting the images together. We have another four months — a short four months.

Barbara Robertson is an award-winning journalist who has covered visual effects and computer animation for 15 years. She also co-founded the dog photography website dogpixandflix.com. Her most recent travel essay appears in the new Travelers Tales anthology The Thong Also Rises.