The leading VFX studio delivers 552 visual effects shots on Oscar-winning director Chloé Zhao’s MCU Phase 4 adventure, including the 3rd act beach fight scenes and planet-sized Celestial Tiamut.
Marvel Studios’ Eternals, the 25th movie in the Marvel Cinematic Universe, introduces audiences to 10 super heroes never before seen on screen. Directed by Oscar winner Chloé Zhao (Nomadland) and produced by Kevin Feige and Nate Moore, the film follows a race of immortal aliens from the distant planet Olympia who came to Earth thousands of years ago to protect humanity from alien predators known as Deviants. Alerted to the threat by the Celestials first seen in Guardians of the Galaxy, this group is forced to reunite in order to defend the planet once again when the Deviants mysteriously return.
Working under the guidance of Zhao, Feige, Moore, executive producer Victoria Alonso, and production VFX supervisor Stephane Ceretti (Ant-Man and the Wasp, Doctor Strange), leading VFX studio Weta Digital once again was tasked with key 3rd act visual effects production, something they’ve provided on numerous MCU outings including the last two Avengers films.
Weta’s team was led by VFX supervisor Matt Aitken, a long-time studio veteran who, among other films, was the studio’s VFX supervisor on both Avengers: Endgame and Avengers: Infinity War, and animation supervisor Sidney Kombo-Kintombo, also an animation supervisor on the last two Avengers films.
Eternals not only brought a new set of characters, storylines, and canon to the MCU Phase 4, but with Zhao at the helm, an entirely new visual style. Weta started work on the project back in July 2019, ultimately delivering 552 shots. The scope of work included full CG shots; combining artistic matte paintings with plate photography; helping the filmmakers discover the final design for “cosmic energy;” an erupting volcano; keyframe animation of Kro; new water sims that integrated CG water seamlessly with waves captured on-set; key beach scene fight previs / story development and final VFX; and creating enormous creatures like Celestial Tiamut.
AWN recently spoke to Aitken and Kombo-Kintombo about their work on the film, including the big 3rd act beach scenes and strategies on how best to integrate full CG characters and environments into live-action footage shot on location in the Canary Islands.
Dan Sarto: In broad brushstrokes, what was the scope of the studio's involvement on the film?
Matt Aitken: Definitely the third-act battle, which is an area of Marvel films we often find ourselves in. So, everything on that volcanic island towards the end of the film, all the way through Tiamut rising and being turned to marble. [We also worked on] the whole ancient Aztec city beat in the first act, which culminates in the Eternals disbanding and going their separate ways. It's when we first see Thena go Mahd Wy'ry. But we also picked up all these other little sequences here and there, partly because, with COVID, the delivery date pushed out, and other vendors on the show had conflicts, and we were just able to continue working on it through to the very end.
We did a little bit on Babylon and the ice cave where Deviants are kind of transforming after they kill Ajak. We did Makkari running around the Earth looking for the point of the Emergence. We did Centuri-Six when Arishem is giving some backstory to Sersi and, on another planet, we see alien dinosaurs and Deviants. We did the mid-credit scene with Eros and Pip the troll, the fan favorite, and other bits and pieces as well. It was quite diverse. I think we ended up doing about twice as many shots as any other vendor on the show.
DS: That happens with Weta a lot. Once you get going, you tend to pick up more work along the way.
MA: [Laughs] We just don't like to say no to Marvel, especially Victoria Alonzo.
DS: Weta is as steeped in Marvel Studio feature film VFX production as any other studio. But Eternals represents a new MCU phase, with Chloé Zhao at the helm for the first time, bringing with her a different filmmaking style than we’ve seen in these films. She’s known for her wide-angle lens use and deep focus. How did you integrate the world of full-CG shot creation with her visual style?
Sidney Kombo-Kintombo: It was challenging. It was challenging just because of exactly what you said. It's different. Action shots so often rely on Dutch cameras, long lenses, to feel claustrophobic, that type of stuff. We are versed in that. We understand those things. But when we were first asked to help put together some previs and develop some action scenes, that was the first hiccup. "Oh, okay. She doesn't want a Dutch camera. How are we going to sell the dynamism of the action?"
So, it was really interesting to wrap our minds around Chloé's style of shooting. How to adjust the way we were designing the action so that it worked together with the wide lenses? The results were surprising to us at first. All of a sudden, the motion becomes weirdly more elegant and gives you more depth. We compensated for the depth that the wide lens was giving us... which was different than the usual breaking of the world that you have in the other action stuff. They [the characters] are not breaking much stuff, but it seems they go very fast, then you see them in the open... it helps compensate for what we felt like we were missing with the different style of shooting, if that makes sense.
MA: A big part of it was making sure that we were authentic to that language. Chloé, through her more art-house films like The Rider and Nomadland, has a distinctive visual style that she brought to bear. I think she calls it anthropomorphic camera work where the camera moves in a way that's in keeping with the way people move. So, there's a naturalism to things.
In the beach fight, I think roughly 43% of the shots are either fully-CG or predominantly CG, just because of the way it worked out in post. But we didn't want that to be apparent. We wanted to preserve that visual language so that as we're cutting in and out, people are thinking that they're still there at the beach. They went all the way to the Canary Islands, off the coast of Western Africa, into the middle of the Atlantic Ocean, and filmed on a really wild, quite inhospitable beach. Sidney and I know because we were there with them [laughs]. They went to all that trouble and got a really great sense of authenticity from that location, so we had to preserve that or else it would've been jarring.
DS: With the plates you got from that location shoot, how did you decide where to use full-CG environments like you’ve done so often in the past Marvel films? For example, the beach scene at sunset, with Tiamut in the background. How much CG was used?
MA: Well, the sand really... you've got a film crew walking on this beach... and they had art-department people coming in with brooms trying to sweep out the footprints between takes. But then you just end up with a broader patch of dark sand. So, our shader department came up with a really great shader for the sand. There's a mixture of black iron sand and more brown, rocky sand particles that create these sweeping patterns. We essentially just replaced the sand with CG-sand in pretty much every shot. It wasn't so much a paint out. It was like rotoing the characters off the beach, having a CG model of the beach, with CG water a lot of the time, and using that shader which had dials for color and the amount of silicate sparkle in it.
Then, because we've replaced the beach, we've got our characters standing in the middle of... there's no footprints leading up to them. We could have done a simulation solution, which would've been quite painstaking. But instead, our comp supervisor came up with this fantastic workflow where the compositor could just project a line in a top-down view in Nuke for where the character would've walked up to the point they were at. Then there was a 2D solution for projecting a footprint pattern onto the CG sand. So, in comp, we were able to quickly create paths. There was a sense of the beach being disturbed in a natural way. That was a big chunk of the work that we did.
The second part of the battle ends up on a beach that was completely underwater at high tide twice a day. Not only did that create some scheduling issues for the production, but it also meant that... because they're these big waves crashing on the beach, it changes shape. We got great LiDAR of that beach, but when we did some early match moves on our CG-terrain, it was like, "Hey, they're about half a meter off the sand. There's something going wrong with our camera track here.” But actually, the beach had changed that much between when the LiDAR scans were taken. So, we had to come up with a workflow to generate terrain from the plate because all we really had was the plate and maybe some witness cameras. The terrain became a shot-specific task. I would really advise a production to think twice about filming on a beach if they're ever considering it again.
DS: Sidney, from a story development standpoint, describe some of your previs work and how that helped the director determine what could be filmed as opposed to created in CG?
SKK: They came to us prepared with a story they wanted to augment. One way [producer] Nate Moore was talking about Makkari, for instance, was, "We are saying that she's fast and we have a glimpse of her being fast, but we haven't seen her being fast in a fight." That was important information for us in trying to figure out how we could develop Makkari’s character in those fight scenes. They would mainly shoot in the Canary Island. As you know, it's superhero stuff, so the characters are going to do more than what any human being is capable of doing. So, we knew that we would go CG there.
The first thing they wanted to see was everybody versus Ikaris. How could we plus that action? So, we started there. I got my paper and pencil and storyboarded an idea for a scene that we pitched to the studio. When they were happy with the idea... they gave us some good feedback in term of, "Let's make sure that there is a moment when you see all of them versus Ikaris" type of stuff… we integrated all of that into the storyboard. When they were happy with the story, we went and started to previs it.
Now, when we previs that type of stuff, we treat it as one shot. It's an action beat. We want the action beat to be believable. Then on top of that, we start cutting with different cameras to make sure that the angles we’re hitting are the most dynamic for what we want to tell based on the storyboard. A really gratifying moment for us, once we had the previs, was when Chloé saw that, and her first reaction was, "Yay. I have my Marvel movie." It was good to see her there, because we knew at that moment, we were hitting the mark, not only for Marvel and the way Marvel wants to tell the story, but for Chloé as well. We were respecting her filmmaking style and that was very important for us in the collaboration moving forward.
MA: I remember when you showed that idea where one of the ways that Makkari can create an impact on Ikaris is by coming at him from lots of different angles really fast. She ends up smashing him into the cliff. They just loved that. They hadn't thought that was something she could do. Hard shots to pull off, thank you, but really exciting.
DS: I understand that Kro’s animation was keyframed and based on Bill Skarsgård's on-set performance as well as his external musculature and build.
MA: We got two pieces of concept art from the look-development team at Marvel for Kro. They said, "This is a starting point. We're not 100% happy with this so continue to explore." This is what we call Kro Phase 3, which is Kro after he has evolved, firstly through the death of Ajak, from being almost wolf-like, to being almost bear/ape like. Then, in the jungle, after he's killed Gilgamesh, he evolves into his more humanoid form. We didn't do that second evolution, by the way. That was ILM. But we did all the Phase-3 Kro stuff. All those shots of him fully-evolved.
I remember a review where one of our senior modelers came up with nearly 30 different faces for Kro, quite different, and all of them with hero geometry. She spent quite a lot of time working up these different looks. We presented them to Chloé and Nate Moore and Victoria... I think Kevin Feige was in that one as well actually. We talked about what aspects they were really responding to. The notes from that review ended up informing the face and head that we settled on.
There's this idea that his lower legs hadn't fully finished evolving from the wolf kind of origin. They still had that elevated heel like a wolf or a dog. Then there was the idea that he was basically made up of bundles of muscles. We looked at all sorts of reference. We looked at the boar in the opening scene in Princess Mononoke. Our rigging team was doing quite a lot of work with secondary motion that layered on top of the work with the tentacles that Sidney and his team did.
Then, Bill Skarsgård came in, more for the dialogue stuff, like when Kro's really interacting with Thena, particularly in the cave at the beach. That scene where he taunts her with the memories of Gilgamesh and ends up getting sliced and diced for his troubles, Bill was there. He didn't wear a motion capture suit for his body, but he had his face dotted up and he wore head mounted cameras. We used the footage from those head mounted cameras as reference for his facial performance. But as Sidney will tell you, we didn't solve his facial performance. That was entirely keyframed.
SKK: His face and body were keyframed. Every time you see Kro, it's fully made up just for the purpose of the scene. There are some moments when he's in the cave and talking with Thena, as Matt mentioned, where we used what Bill did as a reference for the facial animation. But it was fully keyframed to make it work. All his action, his fighting style, same deal. We tried to develop that considering his height. This guy is supposed to be nine-feet tall. He's taller than Thanos.
DS: What about the challenge of dealing with scale and huge characters? Celestial Tiamut is enormous. How did you approach showing his size in a realistic way?
MA: We employed a very creative use of scale because, anything physically accurate would've been impossible and also too destructive. They wanted Tiamut's emergence... the damage from it to be localized. I think at one point, we worked out that as he was rising, the tips of his fingers were moving at about 1500 miles per hour. So even though he looks slow, he’s not because he's so big. We definitely had to play with scale so that he felt big. Tiamut is 300 miles from head to toe. But we also needed to tell the story and see the action. So, he couldn't be too lost in the atmospheric mist. We had some issues with precision, just rounding-error issues, because of his scale that we had to circumvent. Sidney, I think you basically just animated him at the same scale throughout.
SKK: Yeah. He was animated same scale and in the most boring possible way. Thanks again to the language of the camera, what was interesting with Tiamut was trying to find the right angle, the right staging for the camera to make the character believable and alive, even though he was barely moving onscreen. We always tried to place our cameras in a way that you could identify the scale at a human size. You had the hand going through the clouds, or beyond them, the hand against the island, the hand against the wave. We rarely had a position where you could only see his hand, because that doesn't help the scale in any way. Tiamut animation was more a matter of us being convincing with the camera work. So long as you feel that the camera is operated by somebody, Tiamut became believable.
DS: How did your work on this film compare to your work on the last Avengers film? Same types of challenges, or new ones particular to this film?
MA: The thing I found particularly challenging with this one was that we had to come up with a new visual language for everything; the cosmic energy, the way they all behaved, the look of the Celestials… it wasn't like we were building on a legacy of past films, which is the space that we worked on with the Avengers films. [On those] there was new stuff to do there but there was also cannon that we had to observe.
This was very fresh. The adage applies that sometimes you don't finish, you just run out of time. You could carry on exploring those things forever. I'm very happy with what we settled on here. But the other thing is just, for whatever reason, it was an incredibly complex show. There were a lot of moving parts. Every shot had multiple different effects, simulation tasks, digi-doubles. There's environment work. It felt like every department had a lot to do on every shot. To me it felt like the most complex Marvel show I've ever had to do.
Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.