VFX Supervisor Dann Tarmy breaks down his team’s extensive use of water sims, including how they tackled the huge, new challenge of replacing water in plates where the actor was shot in the water, on Netflix’s biographical sports drama of marathon swimmer Diana Nyad’s attempts to swim from Cuba to Florida.
From Aquaman and Midway to The Rescue and Godzilla vs Kong, Scanline VFX is no stranger to water sims, animations, and extensions. But when the award-winning studio was approached to do visual effects for Netflix’s now-streaming NYAD, a biographical sports drama about 64-year-old marathon swimmer Diana Nyad’s attempts to swim from Cuba to Florida, Scanline visual effects supervisor Dann Tarmy saw the opportunity to venture into uncharted waters.
“On Tomb Raider, we definitely had shots where Lara Croft is in the water and we're reasonably close, but I don't think we've done a project where it's such an extensive body of so many shots of this type of work,” says Tarmy. “Scanline has a very long history of doing water visual effects and some of the projects I’ve done in my career were about putting visual effects water into shots and integrating practical boats, or even CG boats, with the CG water. But having to replace water in plates where our lead actress is actually in the water was a whole new challenge. And anytime I get to try something new and different, I get excited.”
And, to think, it almost didn’t happen.
“At the time we were first approached, Scanline was fully booked up,” remembers Tarmy. “And while the work looked interesting to us, and was definitely up our alley, we felt like we couldn't resource it properly to do the work justice. So, we had to walk away. It wasn't until the end of 2022, when they came back to us asking, again, if anything had changed in our scheduling, that we were able to look at our resource forecasting and see that some shows had shifted, as they do. A gap had opened that gave us some bandwidth to be able to get involved. We got awarded about 180 shots.”
A remarkable true story of tenacity, friendship, and the triumph of the human spirit from directors Jimmy Chin and Elizabeth Chai Vasarhelyi, NYAD recounts a riveting chapter in the life of world-class athlete Diana Nyad. Three decades after giving up marathon swimming in exchange for a prominent career as a sports journalist, at the age of 60, Diana (four-time Academy Award nominee Annette Bening) becomes obsessed with completing an epic swim that always eluded her: the 110-mile trek from Cuba to Florida, often referred to as the “Mount Everest” of swims. Determined to become the first person to finish the swim without a shark cage, Diana goes on a thrilling, four-year journey with her best friend and coach Bonnie Stoll (two-time Academy Award winner Jodie Foster) and a dedicated sailing team.
With a slew of circumstances working against Diana’s efforts – from dehydration to predators like box jellyfish – it’s no surprise that a large chunk of Scanline’s 180 VFX shots included sequences where Bening is caught up in a massive storm that nearly takes her character Diana’s life.
“When our production visual effects supervisor, Jake Braver, and I were first discussing the storm sequence, it was pretty clearly presented to us that directors Jimmy and Chai were looking for a more dangerous and threatening storm than what they had been able to capture in the tank,” explains Tarmy. “So, Jake presented us with some reference clips of stormy oceans to get us thinking about how we would match something like those references. How do we make the waters more threatening, without making them deadly? We wanted the audience to feel like people could drown in this storm, but we didn't want them thinking that living through this storm would be impossible.”
Luckily for Tarmy and the artists at Scanline, Braver, true to his name, was “up for anything.”
“All ideas and all approaches would be considered and everything was on the table,” notes Tarmy. “The only caveat was that they wanted to keep as much of the actor's performance as possible, which is not an easy thing to achieve.”
The first step, according to the supe, was to define what this storm would be, determining the size of swells, how big the gaps would be between waves, the extent of the ocean foam, and anything else that would contribute to the tone and authenticity of the scene.
“We also studied the references to focus on the difference between what you see in a storm and what you perceive,” says Tarmy. “Our CG Supervisor, Justin Long, and our effects supervisor, Andreas Vrhovsek, led a team of effects artists to develop all the necessary looks for this ocean that the clients would gravitate toward.”
Running concurrently with the effects development was the asset team, also led by Long, creating animated assets for every person, boat, prop, and anything else that touched the water.
“The plan was still to use as much of the plate photography as possible for all those boats, props, and people, but we knew there would be times when we would need to use bits of the CG renders of the digi-doubles to fill in the gaps where the waves had gotten bigger, or maybe because the shape of the swell was now revealing more of the boat at the bottom than you see in the original plate.”
The ocean development and the base ocean was created in an internal, proprietary software called Flowline while Tarmy and the other artists used Houdini for a lot of the sea spray and rain. Their actual close-up rain and rain interactions were done in Nuke due to the flexibility to iterate quickly within comp instead of constantly going back and forth between departments.
And that’s just the effects development.
“Once we have all these assets ready, everything has to be roto-animated,” explains Tarmy. “Everything has to be tracked into the scene and everything needs a tracked camera to match with the object so that if you move the object, you can project the plate onto the CG doppelganger. That lets us keep whatever motion was on the original element, as well as whatever new motion we wanted to add to it.”
This also means the filmmakers don’t have to reshoot anything on set. If the team reanimated a boat, the projection camera went with it, and Tarmy and the crew could still project the shot plate onto that new animation. Nevertheless, it’s a lot to layer together.
“When we got into shot work, one of the things we did was have our effects team create a 400-frame mesh of the new stormy ocean and the layout team then used that new mesh of all these ocean waves to place them in every single shot, so that we didn't have to start every shot by doing effects sims,” shares Tarmy. “We would use that base ocean to define the direction of the waves and the timing of the waves. It’s all in layout in Maya. And, if the director said, ‘Can you have that wave hit the boat 15 frames later?’ we could offset it and get the right timing for the ocean.”
Of course, once the ocean movement was changed in a shot, the movement of Diana swimming in the water, and the movement of the boats, no longer worked. “That meant we needed our storm animation team to go in and re-animate everything,” says Tarmy. “And we were still trying to keep as much of Annette’s performance as possible, but where she was in the plate, might now be underwater. So, we would have to relocate her up above water. But now the shape of the waves has changed. So, you might see her legs sticking out of the back of the wave, which of course would never happen. So, we would reanimate the lower part of her body, trying to keep her feet close to the water surface so that when she’s kicking, we can get little splashes. All of this was led by our animation supervisor Christian Kratzert.”
In addition to animating Diana, the boats, oars, and other assets, the team would also have to reanimate the camera to roll up and down with the waves and match how this new stormy ocean would appear if was being filmed by a real cameraperson.
“We did do an initial rig and an initial setup to have the camera follow the waves, but we often ended up straying from it,” admits Tarmy. “First off, since you don't see what's below the camera, it's hard to know exactly what the waves are doing down there. Sometimes we’d get these little weird bumps that were technically accurate to following the waves, but didn't feel realistic. The second thing is, to help sell the stormy ocean, we wanted the camera to go under the water sometimes, like a person would in that situation when a big wave comes through. And any of the automated rigs we came up with didn't have that type of finesse. So we often started with a concept like that, but then ended up hand-animating it.”
Once the team had all the animation squared away, the scenes would go back to effects.
“They basically take the initial ocean development, that initial sim, and now re-simulate on top of that ocean, adding a 3D fluid sim, adding the foam sims, the whitecaps, any interactive splashes from the boats or the people, bubbles and foam under the water surface to get that right scatter effect, surface level spray, airborne spray, mist, rain, rain hitting water splashes, you name it,” says Tarmy. “Once all that’s simulated, a lighting team comes along and renders it all in a way that looks realistic and feels good, but where a compositing artist can still make creative tweaks.”
And all of that has to be done without breaking the renderfarm with billions and billions of hours of rendering. Tarmy describes the process as being like peeling an onion, where, as they tackled one problem, they discovered another layer of issues to solve.
“It drove my production team a little bit crazy sometimes, especially with the sims,” he notes. “It wasn’t a one-size-fits-all with those effects. You couldn't use a sim that was intended for above water and an extreme wide shot with an underwater close-up. And comp had the same issues. I was really fortunate on this project to have a strong team that presented me with solutions, and not just problems.”
The storm was a biggie, but Tarmy and his team also worked on the sequence where Diana and Bonnie witness a demo of how the team plans to keep Diana safe from sharks using an underwater sound system, which eventually evolves into the use of a giant red light tube. They also did the massive effects involved with Diana hallucinating an underwater Taj Mahal with fish of every color swirling around her as she swims just above a marble mausoleum shrouded in light. It naturally involved a hefty amount of animation but, luckily, no raging storms.
However, that wasn’t always a benefit.
“It would be easy to say that the storm was the biggest challenge because there were so many layers to it, but the tricky thing about some of those more calm ocean shots is that there's nowhere to hide,” says Tarmy. “When you're doing a storm, if things aren't quite coming together, you can add some more rain, some more mist, more waves, just to make it messy. And not only does that sometimes help you solve your problem, it might often make the shot better. But when there is nowhere to hide, the integration between your CG ocean and your plates and your actress is critical. It becomes very challenging.”
The VFX crew also faced a big challenge simply in having a majority of the shots they worked with in the film being in a tank, rather than in the ocean.
“We only had four shots done in the real ocean, and they were extreme wide shots where the boat was tiny,” notes Tarmy. “When it came to working with water tank footage, even the close-up water doesn't look like an ocean. At a glance, I think a lot of the audience might not notice. But when you look at the difference of current flowing through the shot, it becomes more obvious. If the ocean current is supposed to be at her back, is it always consistently at her back? Because, in a water tank, water bounces off the sides and goes in every direction. So, the ocean extensions we were initially tasked with ended up being full replacements in every single above-water shot.”
He adds, “The tank also didn't have as much of those little particulates and all the other little stuff that's out there in the ocean. So that was something that had to be added in and then moved with the current at the right speed.”
But despite all the unsimulated sweat and tears from Tarmy’s team, this was a chance for a group of talented effects artists to come together and compile their experiences in water effects work to create something truly astounding.
“One of the things that Jake and I agreed on from the jump was that we had to think about these oceans not simply as set extensions, but as a character in and of itself,” says Tarmy. “I'm not a sailor. I love the ocean, but from the shoreline. But I have friends who are sailors, and if you talk to anyone who spent a lot of time on the water, they talk about how the ocean has moods, and how, on some days, it can be very unforgiving, while on other days it can be very supportive. We wanted to harness those moods and help pair them with Diana's strong will and strong personality so that, when necessary, the oceans could match her level of strength and get across to the audience how difficult of a feat this swim was.”
Victoria Davis is a full-time, freelance journalist and part-time Otaku with an affinity for all things anime. She's reported on numerous stories from activist news to entertainment. Find more about her work at victoriadavisdepiction.com.
