The Australian VFX studio tackles a variety of futuristic graphic elements on Marvel’s latest tentpole feature film.
“We always seem to be brought in to solve these weird esoteric things,” chuckles Animal Logic VFX supervisor Paul Butterworth, who had to work on a considerably bigger scale for Avengers: Age of Ultron compared to its predecessor. The Australian company was responsible for 80 shots where key graphic elements needed to be produced, such as hologram representations of Jarvis and Ultron, various lab graphics, and the depiction of cyberspace as a 3D environment. “We start by doing visual research for cool images that might relate to the brief,” remarks Animal Logic executive producer Jason Bath, “such as interesting sculptures, photography or paintings, and then we put together a mood board that could run for almost 50 pages.” Next, the client comments on what elements are applicable and exciting to them. “Once we have narrowed the brief we get a whole bunch of different designers together, put them in a room and basically give them the same loose brief,” states Butterworth. “‘Jarvis is sort of orange and box-like and is made out of code or data.’ We let them run wild and see what happens. We usually allow about a week or two. We collate those as another round of visual research and give the materials to Chris Townsend [the film’s overall VFX supervisor]. He’d handpick the ones that might be relevant and also resonated with the director Joss Whedon. They’d come back and say, ‘This is cool. Maybe we could make this more like this.’ We hone it and get more focused with the key artists and work down to a couple of key concept frames that describe the character or thing.”
“Visual effects look great in dark environments but Chris likes to challenge us,” notes Butterworth, who had to deal with a set that had supermarket-level lighting and bright shiny surfaces. “The set environment had a nice high-tech stark laboratory feel but it’s an absolute nightmare trying to put something that is glowing in front of that because essentially it disappears. What we had to cleverly do was to subtly underexpose areas directly behind the hologram so you would have a sense of it popping out. We rendered a lot of different passes of reflected holograms on different metal surfaces for proper comp integration. Lastly we would do a lot of relighting of the actors. One of the other challenges we had, is to make the eyelines of Robert Downey Jr. and Mark Ruffalo work. They don’t have anything to look at on set and sometimes they’re looking all over the place in the preferred take. We often go through some tricky design and layout iterations so in the end you go, ‘How did they act to that!? That’s amazing!’”
“This is the first time you’ll see Jarvis as a highly sophisticated holographic operating system that you can walk around,” notes Butterworth. “Not only was he an object that you could look at that has movement and shape but you also needed to see him talk and immediately get the sense of a complex intelligence. We played with Jarvis being made out of this rectilinear type of object and with modules sliding within his volume. This look was quite advanced but then the studio decided it was not the right direction for his character. So we redesigned Jarvis with a spherical base that had a central heart and these weird rings rotating around him. Code and data is moving between these different ring layers.” Motion played a big factor in the design process. Butterworth continues, “Once we had a bit of a look it was about doing multiple motion tests to see what looked cool. We presented them to Chris and the director and ask, ‘Is this how you see sound moving on Jarvis? Is this how his rings would move around when he is calm or excited?’ It was those types of things you were trying to solve. In the end we rigged Jarvis such that his overall motion could be driven by dialogue waveforms so when Jarvis talks the data vibrates and moves and the rings animate differently.”
“Tony Stark [Robert Downey Jr.] has used some of the technology which is embedded into Loki’s sceptre to create an artificial intelligence based on his own Ultron peace-keeping program,” explains Butterworth. “You’ll see a holographic representation of that sceptre technology [Ultron] as well. This needed to be considerably more sophisticated, crazy, larger, and ornate than Jarvis.” Old Sumerian languages were studied for the alien text and code. “In starting to develop this we asked, ‘How does Ultron fire on? Stark will take his little phone device and flick it, triggering particles and then grow the structure. What does that look like?’ You do a test for that. ‘How does Ultron grow?’ He grows on from the center out and some of the shells in his design need to activate. ‘How does his surface move around? Is it spiky or is it ball-like? Is it always evolving?’” According to Bath, “We did tests on how the surface of Ultron moves. It has a fractured angular quality to it and there are little pops and pings in its motion. They liked that in the tests which surprised me as I thought they would have gone for a smoother animation. But the Ultron hologram is a complex structure of code after all and Joss and Chris didn’t want to go too organic.” Cause and effect was an important element. Bath continues, “How that energy flowed over that surface was driven procedurally. If data moved in a certain direction it affected the shape and motion of Ultron. It gave you interesting results.”
Depicting digital entities traveling on the Internet was another key area of visual design. “One of the first things we talked about with Chris was creating the Internet,” reveals Butterworth. “What does the Internet look like? Is it code? Is it a really big library? How do you move through it? It’s something that is so familiar to all of us yet is really hard to define, especially visually. We knew at some point Jarvis and Ultron would be in cyberspace and there would be the need for Ultron to search for information. We referenced ideas from The Matrix  and other similar films but early on Marvel was drawn to a strange maze-like structure that you could wander through but could easily get lost in, not knowing what was up or down or left or right. There needed to be a sense of claustrophobia about it but then it could be open. Having the cyberspace environment in motion was a key factor. What we did was look at fractals and this idea that it didn’t matter how close you got to a fractal structure there was a sense of complexity to it. We said, ‘It’s an interesting idea for representing the Internet because if you’ve got gazillions of [pieces of] data, what could they look like at a bite-size level?’ It could be making these wild constructs that could form, grow and evolve. Wrangling that was incredibly hard.”
A script change altered the initial idea for the sequence. “It became more like Ultron would dive into cyberspace to manipulate something or search for some information,” reveals Bath. “Then it changed to become where Ultron gains consciousness. It’s literally a birth. He wakes up and there’s nothing. As he become more aware more is revealed to him until it becomes overwhelming.” Butterworth adds, “Very quickly through a series of shots, Ultron finds Jarvis and cyberspace starts to build around him. Jarvis is explaining to him that you’re a software program and your protocol is to be a peacekeeping program. Ultron then goes through a journey of discovery. ‘Who made me? Who is Tony Stark? What’s the definition of peace?’ Ultron starts to gather data quickly - he works out who are the Avengers, what are peace and war, and makes his mind up about his own identity. This is not the identity that Tony Stark was hoping for, which was to protect humans. Ultron figures the Earth would be a more peaceful place without humans on it. Jarvis sees what’s happening and the two holograms faceoff against each other. Ultron kills Jarvis and builds himself a suit. That’s one small section of the movie which is about a minute long.”
How does one visually represent a computer operating system murdering another such system, as is the case when Ultron kills Jarvis? “With those types of questions I go, ‘I have no idea but we have to come up with something real quick!’” laughs Butterworth. “Ultron has these data streams that grow panels of code and we all liked the idea that he uses the multiple data streams to form spear-like appendages. You’ll see early on as he’s growing, ‘That’s kind of pretty.’ But later on when he has gone mad you’ll see them stab and destroy the Jarvis operating system.” Cyberspace posed the biggest challenged as key story elements needed to be solved. “Paul sat down with Chris Townsend, Kevin Feige [the film’s producer], Victoria Alonso [the film’s executive producer], and Joss Whedon, and did a couple of workshops over a couple of days,” remarks Bath. “That was the key to kick off the shot production. We had all this cool look dev for cyberspace but we needed to zero in on what we were going to use it for. Finding the heart of what we needed to convey about the birth of Ultron in such a short space of screen time was incredibly important. It helped us to focus.”
“They have a lot of these glass monitors which seem to be the trend now in Sci-Fi films. They’re incredibly impractical but they look good,” states Butterworth. “Marvel supplied quite a lot of bits and pieces of monitor graphics that were used for the onset playback that we could then collage, add to with our own elements and populate further monitors and graphics. On a couple of specific scenes where Stark and Banner are working together everything was specifically made for those shots. Stark is manipulating something and sends it across to Banner, who has his own experiment pushed off of his monitor. But it’s obvious that Stark has supplied an important piece of what they’re trying to solve. They then move onto a much more complex holographic experiment which was a combination of CG and 2D graphics. We have a lot of fun doing that stuff. Jason and I tend to work closely on these kind of things to try and figure out what the key information is and what the story beats are that need to be solved.”
Cinematic realization of cyberspace was a major concern for Butterworth. “There’s a particular shot where Ultron is looking at the Avengers’ files from his POV and that was the first shot that was sort of rendered and put together. When we saw that put together I said to myself, ‘This is going to work.’ It was incredibly rewarding to find an interpretation of the internet that Marvel loved.” A pleasant surprise for Bath revolved around a key graphic. “Mine was probably Ultron because he was an alien hologram that had to be more complex than Jarvis and that was something you could have been chasing your tail on forever. It came together slowly and steadily but they never went, ‘That’s it.’ We started putting him into shots and they started talking about other things. I asked, ‘Is everyone okay with Ultron?’ They went, ‘He’s cool. We like him.’ I went, ‘I guess he’s done then!’ But we always added improvements if we felt they were needed. We have a good design crew that loves to help solve things and come up with cool stuff. They never stop trying to make things better.
Butterworth enjoyed collaborating with Townsend on the project. “All the credit to Chris, who had a lot to deal with on this massive and unwieldy project. He’s a nice person to work with.” Bath adds, “And very patient. Sometimes when we didn’t get what they were after he would say, ‘It’s OK guys. We’ll go through it together.’ We love the trust that Marvel gives us. As hard or frustrating as this type of work can be it’s a rewarding and enjoyable journey.”
Trevor Hogg is a freelance video editor and writer best known for composing in-depth filmmaker and movie profiles for sites such as the CGSociety, 3DTotal, Live for Films and Flickering Myth; he is a big fan of Raiders of the Lost Ark, Batman: The Animated Series, The Hobbit, Studio Ghibli, and Peter Weir.