Senior previs/postvis supervisor Gerardo Ramirez discusses the highlights and challenges of the fourteenth collaboration between The Third Floor and Marvel Studios.
The Third Floor served as the dedicated visualization team for Marvel’s ‘Avengers: Infinity War,’ creating more than 3,000 previs shots and 3,000 postvis shots, as well as techvis diagrams for the production shoot. All images courtesy of The Third Floor, Inc. © 2018 Marvel Studios.
For Avengers: Infinity War editor Jeff Ford, The Third Floor senior previs/postvis supervisor Gerardo Ramirez has a major role to play in his discussions with Marvel Studios visual effects supervisor Dan DeLeeuw. The trio work together to refine and add images to the plate work before handing them off to visual effects vendors to produce photorealistic digital effects. “The sheer amount of visualization was incredible,” Ramirez comments. “We completed more than 3,000 previs shots and 3,000 postvis shots for Infinity War alone!”
Complicating matters for Ramirez and his team was simultaneously working on Avengers: Infinity War and its sequel, with the two films being at two different stages of production. “We needed to plan ahead and schedule our previs to the shooting schedule,” he explains. “We staggered the sequences between the two films and adjusted the size of our team as sequences overlapped in priority. There were times when we were previsualizing multiple large sequences across the two films while also creating techvis for scenes that would be filmed in the coming days while providing postvis to editor Jeff Ford for the scenes he needed to present to the directors.”
The Third Floor senior previs/postvis supervisor Gerardo Ramirez.
Early on, Ramirez looked for ways to help maximize the time artists spent on the creative part of the process: layout, camera, timing and animation. “One of the techniques we used regularly was to create rough blocking pass edits,” Ramirez recounts. “Artists would first do a rough, untextured version of their shots that I would then cut together with our previs editor. In the edit, we would experiment with the cut and timing, sometimes creating indications on timing of events within the edit. The artists could quickly see how their shots were being used and could focus their animation, lighting and effects on those frames. This technique helped speed the process because the artists’ CG scene files were light and fast to work in since computer-intensive processes like lighting, textures and effects only happened after the shot was proven to work in the edit.”
“One day we might have been focused on ways for Thanos to fight Iron Man on Titan, and suddenly we’d go to thinking about how to create an intimate moment between Thanos and young Gamora.”
The Third Floor’s pipeline team created several tools to help manage and organize aspects of the shot production pipeline, which gave artists more time to focus on the creative process. “The Third Floor’s toolset includes Autodesk Maya for CG modeling and animation, Adobe After Effects for compositing, Pixel Farm PF Track for match-moving of plates, and Adobe Photoshop for the creation of textures,” says Ramirez. “Our role in previs was to help develop the action and storytelling, including visualizing the big set pieces and fight scenes. We would work from storyboards, script pages or from beats laid out by the directors. Previs shots would be edited into sequences for review, and brainstorming was always welcomed for ideas that could take existing story beats to another level. From previs, we would often map out technical components and what would be involved in filming the shots. Our workflow also included look development for environments and working on specific facets related to the characters, including the powers, fight styles and gadgets they would use.”
Ramirez had previously collaborated with filmmakers Joe and Anthony Russo on Captain America: Civil War. “In the previs process, they put a high level of trust in our team, allowing us to pitch versions of the scenes based on their main beats. As directors, they were always aiming find the coolest moments and supported a wide level of exploration and brainstorming by the previs team. In the early stages of a sequence, they would ask us to do motion studies, where we would explore ideas for fights moves, new character abilities and team-up moments between characters that hadn’t yet been seen together. Many of the ideas from these motions studies inspired development of a scene in the script. There was also great collaboration with and support from the whole filmmaking team, including visual effects supervisor Dan DeLeeuw, visual effects producer Jen Underdahl, production designer Charlie Wood and Marvel producing executives Kevin Feige and Victoria Alonso.”
Extensive concept art and storyboards were provided by the production team. “The challenging part was that as the movie evolved, we would receive material that needed to be updated,” Ramirez notes. “Essentially, each step of the process took the sequences to another level. At times, the story in the scene would evolve so much that it would go back to script pages, then to storyboards and finally back to previs and postvis, each time getting better and better.”
The previs assisted with determining which characters were needed for each shot. “There are 60 or more characters across the movies. Part of the previs was helping define who would be in different shots, what they would be doing and when they would next appear,” Ramirez continues. “The challenge for us was not so much the size of the cast but how the characters would proceed through the story. As artists who have worked across many Marvel movies, we felt a sense of responsibility to help develop the biggest and coolest moments that were worthy of the characters.”
The goal was to have the previs and postvis blend in closely with live-action footage. “Our asset builders used real-world scale, textures and lighting to help make our assets feel real,” Ramirez describes. “We would use scans of the actors or models provided by the visual effects vendors when available to ensure our assets matched as much as possible. Our approach to animation was similar as we wanted to make it feel like the real thing. When visualizing an action sequence, we made sure the animation was realistic and true to the style the directors were looking for. One of the ways we achieved this was by using motion capture for general actions like running, walking, and dodging. We would often use a section of the mocap and then hand key the more specific action. On a movie like this, where so many of the characters have incredible abilities, we hand-animated the story the majority of the time, still also making sure the character and camera animation felt real.”
Quieter character moments needed to be visualized along with key fights and massive battles. “For some scenes, there was plenty of time and description while for others, we had to experiment in a shorter amount of time,” remarks Ramirez. “There were a couple sequences where we had storyboards that were edited into an animatic. For those scenes, we used a more traditional visualization technique where we animated the individual shots based off of the storyboards, making adjustments to the action or edit as the sequence developed. Some scenes were visualized from just a few lines in the script that didn’t necessarily spell out the detailed action. In those instances, our team would brainstorm ideas as a group and identify actions that would compliment the characters and story.”
“On a couple of occasions, we were tasked to quickly visualize a newly conceived scene that was going to be filmed soon, sometimes just days away,” continues Ramirez. “For those instances, I would have the artist create still frames instead of fully animated shots to essentially produce ‘previs storyboards’ that also included real-world scale, layout and camera information. One of the most satisfying techniques we used to iterate exciting fight gags was to have each artist animate what they thought would be the coolest five-second fight scene between the required characters. Artists put forth their best ideas and from those, we picked the best ones to work into the previs. This allowed us to mine for great ideas and give artists a chance to offer ideas to shape the film. The directors were fans of this technique as it helped generate a number of iconic moments.”
Hundreds of techvis diagrams were produced for Avengers: Infinity War. “This ranged from simpler diagrams where we indicated position of the previs camera in relationship to character or sun shadow path to more complicated techvis, where we were helping to work out how to film complicated character actions, like flying shots of Doctor Strange and Star-Lord,” says Ramirez. “Our team included a dedicated techvis artist, Andy Bloch, who produced a staggering 300-plus techvis diagrams and videos for the production. One of the techvis tasks was to help figure out how to film Thor and the dwarf king Eitri simultaneously for scenes in Nidavellir. The directors wanted to ensure that Chris Hemsworth and Peter Dinklage were able to act off of one another during the shoot, so to do this they needed two camera setups. The camera for Peter would be closer and lower to the ground to create appropriate scale when composited to the Hemsworth plate. We were tasked with preparing the technical breakdown for the cameras to achieve this look.”
Time of day studies were consulted to determine the best lighting positions of particular shots. “Some of the techvis we created helped to model and predict the level of the sunlight and shadows that might be expected for the shoot date,” reveals Ramirez. “For the scene with Thanos and young girl in the pagoda, the directors wanted to make sure that the characters were lit from behind by the sun. To help achieve this, we modeled the pagoda on a virtual backlot where the physical set was going to be built. Using this model, we were able to identify the orientation that was needed for the set so the sun would be in the right spot on the date that scene would be filmed. For the Wakanda Battle, we also did studies to show what time of day shadows from the tall trees would begin to spill into the sets. This helped them schedule certain shots to avoid shadows.”
The Third Floor evolved its postvis pipeline to bring CG character comp data in from the set. “We were able to take motion capture data provided by the team at Profile Studios, ingest it into our Maya scenes and quickly populate the postvis with character performances that had been recorded. Editorial would cut in the postvis with mocap and essentially look at the take in context.” The varying sizes of the characters ranging from Thanos to Rocket needed to be taken into consideration. “We spent time studying and planning for the character size relationships. Much of our techvis was centered on helping solve for height differences for the shoot and finding clever ways to position the actors, props and cameras,” Ramirez says.
“One great thing about the collaboration on this movie was that we were able to help visualize not just big battles but also pivotal story points and moments that were more intimate and not huge scale,” he observes. “The battle in Ebony Maw’s Q-Ship was one that had various iterations. It started off as a massive battle that took the characters into various parts of the ship. However, given that we just came off of a huge battle in New York, the directors chose to focus the sequence more on the characters and their interesting relationship. The Wakanda battle has a high level of emotion and also many characters. It was a scene that had to hit all of the high marks. We found ourselves also visualizing quiet character moments that were executed very close to the previs. Having worked on many of these projects, it was crazy kind of tearing up doing the previs on characters that we love! The Titan fight was another sequence that was quite involved for previs. We worked from an outline of the action that was refined and defined all the way through to postvis.”
Staying on top of the shooting schedule was the biggest challenge. “We needed to be able to quickly adjust our workflow to the needs of the production,” says Ramirez. “There were times when a sequence that had been months away in the previs schedule suddenly moved up. This meant we would shift gears, quickly prep and model the assets and shift our creative minds as well. One day we might have been focused on ways for Thanos to fight Iron Man on Titan, and suddenly we’d go to thinking about how to create an intimate moment between Thanos and young Gamora.”
Digital assets were shared with the various departments. “We would regularly send our previs models to Profile Studios for use in motion capture sessions. We also shared assets with the art department. They would send models when designs were updated and we would then incorporate them into our previs environments. As action developed in the scenes, we would work with visual effects supervisor Dan DeLeeuw to adjust the layout or design and then send the updated models to the art department so they could make any adjustments necessary to their sets,” Ramirez says.
“The Titan Battle was something I was looking forward to see on the big screen,” he remarks. “It’s the moment fans have been waiting to see since the first Avengers film: Thanos battling our heroes. Also, our artists helped develop many of the actions in that scene, so it has a special place in our hearts.”
Avengers: Infinity War marks the fourteenth collaboration between The Third Floor and Marvel Studios. “Working on this project was very personal for my team and me,”Ramirez says. “Many of us have been working on Marvel Studio films for several years now. We’ve grown with these characters over the course of each project and Avengers: Infinity War is the culmination the Marvel movies we’d been working on for many years.”
Trevor Hogg is a freelance video editor and writer best known for composing in-depth filmmaker and movie profiles for VFX Voice, Animation Magazine, and British Cinematographer.
