Jerome Chen Talks ‘The Amazing Spider-Man 2’

With a strong worldwide release including a $91 million plus opening weekend at the U.S. box office, Sony’s The Amazing Spider-Man 2 is well on its way to blockbuster status. The second edition in the studio’s webbed superhero reboot, helmed again by director Marc Webb, pits our hero against old and new foes alike, with much of the ensuing mayhem taking place within one of the most visually iconic and familiar city settings in the world – Manhattan.  

Veteran Sony Pictures Imageworks visual effects supervisor Jerome Chen is also back again, as is much of his creative team. I recently had a chance to talk to Jerome about his work as the overall visual effects supervisor on the film. He shared his insights on the film’s extensive use of previs, how he approached breaking down the CG production requirement and just how inherently difficult it is to wrangle the visual effects production on such an enormously complicated film. 

Dan Sarto: Can you breakdown for us your main responsibilities on this film?

Jerome Chen: I was the overall visual effects supervisor for the movie. Though I’m an employee of Imageworks [Sony Pictures Imageworks] I was loaned out to the production to oversee the entire project. Imageworks did about 1000 shots. I designed things so that anything centric to Spider-Man specifically would be done by Imageworks, since they had the assets already. Our director Marc Webb really loves the style of animation that Dave Schaub [Imageworks animation supervisor] has created and brought to a new level in the new film.

When you plan a show, you always want to be efficient. The studio wants you to split it up as much as possible. But from a logical point of view, anything that dealt with Spider-Man really made sense to keep at Imageworks. So that also included Electro and Rhino. The other anchor company was MPC in Vancouver. They do great animation. They have great compositors. They did the eel sequence at Oscorp, the tank effects, the tank exploding as well as the airplanes.

Blur Studios has a great design department so I had them do some special projects shots where you see the Dock Ock arm and other things like that in the background. 

DS: I thought I saw Tim Miller and Blur listed in the credits. I haven’t talked to Tim in a number of years.

JC: Blur also did the end title sequence. Tim and I knew each other in Washington, D.C. before we moved to LA. 22 years ago. I’ve known him for a long time. I really love his creative sensibility and his team over there.

DS: So what were the main sequences on the film?

JC: Times Square, conceptually, is a very easy idea. Spider-Man and Electro battle in Times Square [laughs]. Sounds easy. But when it came down to the practical planning of trying to deal with Times Square…what I did was I read the script and then met with the technical leads and key artists at Imageworks and said, “OK, we have a fight in Times Square. We have Electro firing off electro-blasts. We have Spidey jumping around.” So everyone said, “OK, we’ll shoot in Times Square…Right?” And I said, “I think we can. Maybe a little bit. Let me talk to Marc.” Marc talked to the producers, they talked to New York and New York said, “No, you’re not shooting in Times Square…Well, we’ll give you one night.”

So, we shot one night in Times Square and for the rest, we built a piece of Times Square, surrounded it with greenscreen and put in Times Square to such a point that no one actually realized that’s what we did. I thought that would work fine because most people wouldn’t understand why we didn’t or couldn’t shoot in Times Square, so they would just figure we were actually there. That’s how I often judge the difficulty of effects work. Audiences know Electro obviously is going to be an effect. They know Spider-Man swinging is an effect.

The power plant battle was going to be mostly CG so I knew that was going to be a lot of fun. Dave Schaub could go crazy and create any epic action we wanted to do. Patrick Smith over at The Third Floor played a huge part in helping us design the action sequences. He collaborated heavily with Marc and came up with some great, great stuff.

Then we had the end clock tower scene. Not only did we have fighting in the clock tower, but we had to create the clock tower as well. That was a limited set build and a lot of CG work. I knew our digital versions of Gwen and the Goblin would have to be flawless or that scene wouldn’t work. They were going to be called upon to do things too dangerous for real people to do. We had a lot of digital Gwen in there.

We’ve been in business for over 20 years. I think Rhino is one of the first real mech robots we’ve done. Other studios have done Transformers and things like that. So finally, our animators got a chance to do one and they went all out on it. Rhino turned out great.

DS: How did you handle the digital double work?

JC: We did full scans of the actors. We did facial casts to get pore detail. We reworked our shading technology from a rendering standpoint to revise how it handled the skin. The CG versions of Gwen’s hair were pretty difficult to do, particularly blend on the rend [blending on the render]. We took her coat and made sure that matched the scanned matte photograph. We did acquisitions of Emma Stone, Andrew Garfield, Dane DeHaan and Jamie Foxx.

DS: The use of previs, techvis and postvis seems to be on the rise at the major studios. How much previs did you use on this film?

JC: I like to have a lot of previs. As early as possible, I like to have as many questions posed as possible so we don’t get surprised later on. That’s a function of the fact postproduction times are getting shorter and shorter. I don’t have much time to recover from surprises. Meaning, new shots, new sequences, new ideas. So, I want to explore as much of that as possible early on.

The way we do previs on the Spider-Man movies, we don’t just do a pre-designed sequence right from the storyboards. We will do many, many versions of certain shots. It’s almost like we’re creating digital dailies in a sense. We’ll animate something and then we’ll cover it from lots of different camera angles. We give the editor all that footage and he edits the sequence together from that previs. From that, you can see, “Oh, we’re missing a shot here. Put an insert here.”  Or, if they don’t want to do an insert, redesign everything to encompass that moment.

That’s sometimes the hardest part for some directors to get their head around. They have to design a huge action sequence, but where do they start? Sometimes, it’s better to have previs suggest lots of different versions and lots of different takes that you can then edit as if you’d shot it already. You can think of covering the angles with a long lens, with different cameras and crash boxes. You can craft your sequence that way. You show that and it helps the second unit, the DP [director of photography], it helps everyone figure out how they’re going to actually shoot the scene. Or, you determine you can’t shoot any of that and it needs to be all CG. That’s the best place to be if you have previs early on.

For example, the power plant site, we knew it was going to be all CG. We prevised it and gave it to Imageworks really early on so they could start working on it as soon as possible. It’s one of two main scenes I wanted Imageworks to start on early. Times Square and the power plant battle. Luckily, the power plant battle was mostly CG so they could start right away once it was prevised.

DS: So essentially you’re saying that sometimes, when a director doesn’t know yet what they want for a scene, previs helps by putting a lot of different ideas together so the director can start piecing together a scene as they start seeing material they like?

JC: Exactly. You get a previs sequence cut together that works. Then you sit with it for a long time and then suddenly, you say, “You know what? I want to do something different” [laughs]. “I’ve been looking at this for too long.” That’s when you have to turn the director away from the scene. “You know Marc, actually, this scene is really good. You just want to change it because you’re tired of looking at it for the last six months. Plus we actually can’t change it now because it’s too late.” That’s the part you need to deter them from. But Marc makes really good use of previs. He gets the studio really excited by the previs. It’s a way to show the studio, “OK, this is what I have in mind for the movie” without actually having to shoot something. It’s a really powerful tool.

On this movie the entire Times Square fight sequence and all the Spidey-sense shots were prevised. Marc was really compelled by the idea of having this fight happen with all the Jumbotron screens showing the live feed. It was one of the first action sequences the studio saw in previs. That made it even harder because those were effects shots within effects shots. You have to realize, since there’s no set, the image of Electro on all the Jumbotrons, against greenscreens, needed set extensions and electrical effects in his face. That all had to be done twice in a sense.

DS: The complexity is amazing and you wouldn’t even think about it because you’re assuming it’s a real shot of Times Square.

JC: There are 150 Jumbotrons in the shot. Originally the studio wanted eight different angles on the action. I said, “No. That would be eight different effects shots all in one shot. You can’t afford that.” They said, “What do you mean?” I had to explain that every shot within the shot also had to have a digital Times Square in it. Then they said, “Oh yah…” At first, it didn’t make sense to them. But then, when you think about it, you see how that would work and you figure out, “Yah, you’re right” [laughs]. We don’t actually have that footage to display on a digital Jumbotron.

DS: So having that previs not only helps in scene and story composition, but it also helps in planning what and how you’ll do your live action shoots, such as what you do and don’t need to build?

JC: Yah. There really seems to be two types of previs. We sometimes say, “Give us a technical previs…”

DS: The techvis…

JC: Right. If you shoot with a 45mm lens, what’s your field of view? How much of the set do you see? What is the arc of your camera? Those are the things the departments want to figure out for shot planning. That’s not what Marc wants it for. Marc wants creative previs with a pretty high level of animation, including lighting. The lighting doesn’t need to be too perfect. It doesn’t have to look like a finished shot.

But that actually becomes an interesting problem with previs. When you have a temp screening for audience, even for press, they think it’s the real shot because it looks too close. There was an exhibitor screening where they showed the entire Times Square sequence cut together. There were shots in it that weren’t done yet. So they put in the previs. But it looked too close to being real. So, I had to make an even dumber version with Electro on greenscreen against a grey background [so you could see it obviously was not finished]. It was weird.

There’s also postvis now.

DS: Of course…

JC: Postvis drives visual effects producers crazy. They’ll say, “I have this huge previs budget and we should be done [with previs] by the time we’re shooting. Right?” But I tell them, “No. Actually, we have postvis.” They go, “What! You postvis it too?” Postvis goes on for a while as well. So you keep these teams on for sometimes a whole year. Almost the entire duration of production.

DS: But at some point, isn’t the postvis team working primarily with the editor and the director?

JC: Yah. The director is saying, “Now that we’ve shot the plate, this is the action I want.” The editor will order up what they want in a sequence.

DS: Overall, what were the most challenging aspects of this project?

JC: It’s interesting. I knew I wouldn’t have much time when it came to the post schedule. I tried to imagine where I might find myself in the most trouble at the end of the movie if I didn’t start early enough. We got an early enough start on Electro. Electro was really, really difficult. Much more difficult than I imagined. Originally, we were just going to do a little bit of lightning on top of the makeup skin [laughs]. It would just be on the surface. That’s what we talked about early. But once I saw the makeup and we saw some footage [reference] of lightening inside storm clouds, I wanted to something deep inside his skin where the light would filter through. Once we realized we wanted that effect, we realized we’d have to fully RotoMate the face and every shot almost became a full CG endeavor, though we still wanted to keep a lot of the plate. Electro, with all his various layers, was very difficult.

The level we did on one shot never completely transferred to the next shot. So every shot became its own little recipe for disaster. It was hard for the team to attack. So obviously, I knew Electro was going to be a challenge.

Times Square was a giant challenge because…well, everything in the movie was a big challenge [laughs].  You’ve seen the movie. It’s a big movie!

But going back to how I was thinking, the things I was most worried about were the things I wanted to get started on the earliest.

DS: The things you anticipated would be the most difficult and challenging…

JC: Right. That meant I had to pressure the director and the editor to sign off on that sequence early on as well. Times Square was a huge sequence obviously and I wanted them to turn it over to us in April [2013]. We finished shooting in February [2013]. I told them in two months, you’re going to need to give me Times Square to start working on if you want it done by the following April. That was even before the director’s cut was finished. Marc said he’d turn it over to me, but it would be only 80% complete. There were going to be some changes. I said, “Fine.” So, he turned over Times Square and gave us a year to do it.

Actually, one of the last shots to deliver for the movie was the Spider-sense shot of him going up the bleachers in slow-motion, saving all those people. It was one of the first shots started. That took almost a full year to complete. It was really challenging.

I also asked for the power plant to be turned over early. Marc had it all prevised and we all liked the action, so we got started on that. We had to build a full CG set and build Electro. I knew they couldn’t give me the clock tower. I knew that was going to be hard. I knew they were going to mess with that edit until the last second because it was Gwen’s death scene and was so important to the story. So I understood that I couldn’t get that early.  And I asked for Rhino early too. That was fleshed out early on and didn’t change much. We also started working early on our digital doubles. Those actually were dropped in pretty successfully without much headache. I was pretty pleased with those.

DS: Along those lines, were there any sequences where you expected to encounter problems but things actually ended up being easier or less complicated than you anticipated?

JC: No. It was all completely the opposite [laughs].

DS: [Laughs] I’m sorry. I don’t mean to laugh at your pain…

JC: Everything ended up being much harder.

DS: How many of the assets from the first Spider-Man were you able to reuse?

JC: We have a great, mature New York City asset. We don’t have all of New York City. It just looks like it. But we’ve got 65 pretty good buildings now. That asset base has grown over the last 15 years. But there’s always some setback that causes us to have to redo stuff. For example, the previous trilogy, all those buildings were designed for RenderMan. We’ve since switched to the new Arnold renderer so we had to redo all the shaders and all the textures. Also, on this movie, we have a couple buildings that have room interiors. Offices, with furniture. It turns out that Spidey swings close to buildings, and it’s at night, so you need to see something inside the building. Pasting a little picture doesn’t work anymore.

The last movie took place mostly at night. But now, we wanted to do all these daylight swinging shots. We were able to use our New York buildings, but they didn’t look as good in the daylight. So we had to give them new paint jobs with more detail, so they would stand up during the day.

Ironically, the Spider-Man suit, which is the one thing you’d think we can use over and over, gets changed every movie [laughs]. On this movie, they redid it completely. I really like the new costume. I like the bigger eyes. The fabric is a little looser which allowed us to do full cloth simulation to get ripples and folds, fine details that people are saying makes it look more realistic.

So, we were able to reuse rendering technology, animation and just in general, the artist experience. That’s the most valuable thing. It’s not the technology. It’s the artistic mindset. That’s the most valuable thing that we have to use and reuse. We basically have the same team, have the same artists back for the new movie that worked on the last one. They get better and better every time.

DS: Any Simulcam or other virtual production tools used onset for the shoot?

JC: No. We just didn’t have enough time in preproduction to set anything like that up. Marc is pretty good about imagining what he’s going to see. There were a couple places I would have loved to use it. There is a scene in the movie where Gwen and Peter are on top of the Manhattan Bridge. They’re having a conversation, the camera is moving all over the place. All of New York is behind them. I would have loved to use a Simulcam on that scene. We were supposed to shoot that outside to get the lighting right, but it was raining in New York, so we had to move it inside onto a stage with a giant greenscreen world. Nobody had any idea what they were supposed to be seeing. The actors were like, “What am I looking at?” I pulled up the maps program on my iPad and put it in 3D mode. I found the bridge and using 3D maps and Google Earth, tried to show them where they were probably looking. “I think you’re looking at the South part of the island”  [laughs]. The weather messed everything up. It was supposed to snow in New York for a week so they moved the shoot up a week. I was going to try to get a Simulcam setup. But instead, we used an iPad map app and Google Earth.

All that was recreated by MPC, all the New York background imagery. It was one of the more difficult scenes in the movie to finish.

DS: But you would never guess from seeing that scene in the movie how difficult it was to complete.

JC: Yah. You’ve got two people standing in front of 270 degrees of greenscreen. Yet, when the camera is moving around, you have to have the right geography of what part of Manhattan you’re looking at. It needs to look pretty too. You’re looking at the water, at city skyline. We shot helicopter tiles later on and remade a whole world.

There are whole other sections of the movie that no one talks about. Like when Max and Gwen are in the elevator at Oscorp. All the floors moving outside the elevator, the whole interior of the building, that’s 60 floors of all CG environments, complete with moving people [laughs].

DS: Just for background shots of a bit of elevator traffic.

JC: It’s actually 25 shots of back and forth. It was pretty technically challenging, yet it goes by in the movie and most people probably don’t give two thoughts to what they’re seeing. They think you just walked through a shopping mall and shot it. But it’s pretty elaborate.

DS: When you’re onset, how much are you focused just on making sure you get what you need for the postproduction visual effects work versus helping the director craft what you’re actually going to shoot?

JC: I sit up at the monitor with Marc because I just love the process. I love watching him work. If I look at the monitor, I always imagining, “OK, what does this really need to be for the movie?” Marc if great about taking ideas. If he doesn’t want to do it, he doesn’t want to do it. He never discourages new ideas. So I’ll always throw out an idea if I think it’s interesting. You want to do whatever is best for the movie. That applies whether we’re looking at storyboards or early shots. It’s an open environment where I can throw these suggestions out to Marc or Matt Tolmach or Avi Arad [the film’s producers]. If they like the idea, it gets rolled in. If they don’t, they ignore you [laughs]. Having a great collaboration is the best part of making movies. Plus, they all know they can’t do it by themselves. Except for James Cameron [laughs]. Maybe Robert Rodriguez. And Quentin Tarantino. But other than those guys…

DS: Looking back on this film, what part of the project gave you the most personal sense of satisfaction?

JC: This one has been really satisfying. Seeing people watch the film and get lost in the imagery. I’m really proud of Spider-Man and how he flies over the city. I’m proud of the power plant battle with Electro. I’m really proud of the clock tower sequence. It’s such an emotionally powerful scene that relies completely on effects to put the audience right in there without being distracted. Our best work is often stuff that’s invisible.

I’m also really proud that we finished on time. People always say, “We’re not going to finish, we’re not going to finish!” And I always say, “Don’t be ridiculous. Nobody just doesn’t finish a film.” But on this one, I was saying, “You know what, we actually may not finish on time” [laughs]. You know, the power plant battle was unbelievably difficult to do. It was one of the last sequences we completed. We had 50 effects artists on it. I think it was the largest effects team that Imageworks has ever put together. Every shot, you had all Electro’s layers of electricity, plus Spider-Man, his webs, as well as smoke, all the environment destruction, debris flying off where they hit…there’s a lot of things blowing up.

DS: As much as I enjoy watching the finished movies, I love even more talking about how they get made. They’re so unbelievably complicated…

JC: You love the war stories…

DS: Indeed. I feel your pain.

JC: Well, I’m smiling about the experience. But, it’s nice to be done.

--

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.