Sony Pictures Imageworks’ VFX Supervisor Chris Waegner and Animation Director Richard Smith talk about the challenges of working with three Spider-Men, updating a clutch of long-time villains, and ensuring realism in the midst of an action-filled fantasy.
As you may know by now, a modest little film called Spider-Man: No Way Home, which was originally scheduled for a July 2021 release, finally made it to the big screen in mid-December of last year. Currently boasting a rating of 93% on Rotten Tomatoes, the third entry in the newest Spider-Man franchise quickly became the highest-grossing film of 2021 and is presently the sixth-highest-grossing film of all time. Earlier this month, it received an Academy Award nomination for Best Achievement in Visual Effects.
In addition to the multiple complex action scenes, which are par for the course, the challenges for the animation and visual effects teams also involved the need to rethink and reconfigure returning characters from earlier Spidey films, among other considerations. In an extensive and wide-ranging conversation, Sony Pictures Imageworks’ VFX Supervisor Chris Waegner and Animation Director Richard Smith share their experiences working on the acclaimed Marvel blockbuster, and reveal exactly what it took to make it all happen.
AWN: You both have been working on Spider-Man films for about 20 years now. Apart from working in a pandemic, was there anything appreciably different on this film for the team at Sony Pictures Imageworks?
Chris Waegner: As you said, the Spider-Man franchise has been going on for 20 years. We've had multiple directors – Sam Raimi, Marc Webb, and now Jon Watts – and three great actors playing Spider-Man. I think our filmmakers on this one were incredibly passionate about getting our best work on the screen because of the history of this character and this franchise.
AWN: With regard to the returning characters, how much were you able to rely on previous visual iterations, and how much freedom did you have to update them?
Richard Smith: Especially with the villains, there were certain things we wanted to try that were a little bit different. With Electro, for example, the idea that he would move in an instant, basically like a lightning bolt, was something that we didn't really get a chance to do before. But we got to try it in this one. So it was cool, taking that existing character and just tweaking it a little bit, so that we can give you something new, but still respecting the previous iteration.
As for Spider-Man, the body mechanics are pretty similar from person to person. It's all the same kind of physics, whether you’re Tobey Maguire, Andrew Garfield, Tom Holland, or one of their stunt performers. When you’re in the air, or landing on top of a statue, there are certain things that you're going to do to maintain your balance, you're going to have the same instincts. So we tried to find shots where you could actually read some difference. It wasn’t the easiest thing to do, but I think we were able to get a few of them. We were also able to bring in things from previous movies that people might recognize. Obviously having the characters look different is a huge part of being able to tell who they are. Because, I mean, they're all blue, they're all red, they're all moving really fast.
CW: What Richard is saying really applies to the final battle, the third act. The sequence takes place at night. The tonal values and the suits are all pretty similar. The team went back and looked at how other directors and other actors worked in similar situations. Each one of them has their signature poses, or how they fling a web, how they hop from point A to point B. We studied that stuff and reincorporated it into the performances. Then it was up to the lighting and comp teams to help pop those guys out in the final presentation of the shots, so they weren't just one homogenous Spider-Man.
AWN: Speaking of the big third act, which sequences were the most challenging, or notable in other ways?
CW: The third act takes place on Liberty Island at night. The Statue of Liberty is under renovation. The structural environment in which these characters are swinging around gave us a great opportunity, where they could play off the structure as they're using their spider skills. The other thing that was a lot of fun – for which the filmmakers gave us a lot of creative license – was the new incarnation of Electro. We wanted Electro not just to look different, but for his electricity to be more complex. He’s essentially a giant electrical storm. The pops and flashes of light gave the lighting teams this really interesting dynamic where they could sculpturally light the characters and light the environment. And because we established this look of him crackling lightning, even when he wasn't in a shot, there was still this omnipresent threat, which also gave us the latitude to play with the lighting when he wasn’t in the frame.
AWN: How much did previs figure in your work? Were you responsible for look development or were you working mostly from established visual materials?
RS: In the case of the third act, we had previs for the basic sequence – the three Spider-Man [characters] fighting the villains on the island – but it definitely evolved from there, as editorial got ahold of it and shot their plates and started swapping them in, and playing with their timing. A great deal of it was reworked, just because any little cut that changes where something happens, or the order of a shot, necessitates changing a great deal of it. But knowing what you're aiming for is a great starting place.
Sometimes we would need to figure out a way to adjust the camerawork to make it a bit more realistic. Or we had to slow it down because it was way too fast. Those kind of things. Because if we try and follow the previs too closely, suddenly we have issues with realism. If you don't respect the physics, it's just not going to look real. So it’s taking the concepts and seeing whether you could actually fly a camera like that. I mean, it might be a really expensive shot, or it might mean the camera operator jumped off a ledge to their death, but you would know that a camera actually could accomplish that goal.
AWN: Was there anything that was supposed to be produced with live-action plates that ultimately had to be done digitally?
CW: I think that happens a lot with films like this. The client is looking for something visually spectacular, and thinks we can capture it in plate photography. But as the story evolves, and gets more solidified, we start looking at shots more objectively, and realize that in some instances we’re going to have to take out the plate Spider-Man and put in a digital Spider-Man.
For example, one thing a lot of people don't know is that, late in production, after principal photography had finished, the filmmakers decided they wanted to have what we called the “Spider-Man hybrid suit.” That's essentially a suit that Tom Holland wears throughout our whole Liberty Island sequence. That suit is completely digital. Through all of practical photography, they shot him in what's termed the black-and-red suit. What our team had to do was essentially Rotomate every single shot of Tom Holland. He's essentially a head with a digital body. And there are a lot of things that happen in plate photography – wrinkles, muscle twitches, just simple, subtle movements that Tom makes. And we had to replicate all of that in a digital suit. It was very tricky.
AWN: So would you say that was the biggest challenge for you, or were there other complications that required a major effort on your part?
RS: I would say, in general, trying to stay nimble as they're working on the story, while continuing to deliver the shots at a high level, was the biggest challenge. Just trying to help them fill those holes as they come up. It's definitely a challenge, but it's also the fun part. Also, as I mentioned before, trying to make sure we kept everything as realistic as possible. If you're running around and you've got all those bars, what would you do? What is the most practical thing for Spider-Man to do in that situation? So we’re looking for those little moments that might add a bit of extra grit that make it feel more realistic.
CW: I think films of this scale are just full of challenges. In this case, working with the three Spider-Men and the cast of villains that came back – I mean, what a responsibility to the fans and to the filmmakers, to make it look as awesome as possible. Also, the one thing that sometimes gets lost is the hundreds, even thousands, of artists who work on projects like this. Everybody's putting their heart and soul into it to tell the best story possible.
RS: I'd say the same thing. All the people who worked tirelessly to get it done are the ones that really need to be acknowledged more than anything else. It's tons of work and they did a great job.
Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.
