2D Animating the VFX Way on ‘Son of Zorn’

All images ©2016 Fox Broadcasting Company.

While comparisons for Son of Zorn inevitably fall on He-Man for its eponymous warrior hero, or Who Framed Roger Rabbit for the show’s melding of live-action and seemingly 2D animation, neither do justice to the show’s intense, complex production process operating behind the scenes.

We’re five episodes into the hilarious first season and though plaudits are deservedly being heaped on creators Reed Agnew and Eli Jorne, and executive producers Phil Lord and Chris Miller, the show’s raucous humor and pleasing aesthetic owes much to the animation expertise and guile of Titmouse, Inc.

Following his interview with Eric Appel, director of the series’ opening three episodes, Dan Sarto joined Titmouse producer Jennifer Ray, on-set Animation supervisor Barry J. Kelly and animation supervisor, Juno Lee, to dig into the details.

Dan Sarto: First off I must say, I really, really enjoy the show. Can you introduce the project and Titmouse’s involvement?

Jennifer Ray: Son of Zorn is a very high profile project for us here at Titmouse. We’ve been working on it for over a year. Fox approached us to do the presentation pilot, which was sort of the proof of concept short that got the show picked up. Generally speaking, we try and really curate the artists and the leadership for each project that we’re on.

DS: What is your job as a producer on this show? How would you summarize your role and your duties?

JR: Once we get into a series production, I’m working in tandem with the producers on the live-action side, and particularly the post-production producers on the Fox side. We’re all in charge of sort of bringing Zorn to life, making sure what we do works for the other entities. It all, of course, has to do with budgeting. How do we make a show that’s this complicated, that’s never been done before, with this tight turnaround within the schedule? How do we facilitate as much time as possible to give each artist the tools they need to do the best work that they can?

The schedules are happening so quickly, we were shooting an episode a week, which is almost unheard of in terms of planning for animation. Barry and I were in pre-production meetings before shooting began. I felt it was my duty to speak up on behalf of the animation portion because there’d be decisions made that we wouldn’t really feel the effects of for three to four months after shooting had wrapped. It would be about trying to anticipate as much as possible, and applying our knowledge of our own internal pipeline. What issues can we prevent? How can make sure we get shots a certain way to help facilitate the best possible animation?

DS: Barry and Juno, what were your roles?

Barry J. Kelly: I was on-set animation supervisor, so anytime there was a Zorn scene, I’d be there. Live-action footage is shot so much faster than any animator can animate, so I was trying to do rough boards and figure out rough acting and blocking. We had a six-foot tall, comedic improv actor named Dan Lippert who stood in for Zorn and read the lines. I’d be the guy on set figuring out how Zorn would pick up the coffee pot, telling the actors, “Zorn is this big,” or, “He’d be sitting here. Look at this green X and act in front of it.” I would draw rough thumbnails that would go directly to post because editors were getting the footage the next day to start cutting it and all they had was these empty plates with no character.

My job needed a bit of VFX knowledge, and animation knowledge, like acting and drawing, and just a weird general knowledge of tidbits of different animation. My past jobs fit the show perfectly.

JR: Barry wore a lot of hats on-set. If he were in a meeting with me going over next week’s script, he’d be running, literally, from the set where he’s shooting this week’s script. When he wasn’t drawing rough poses of Zorn to send to editorial, he was working with the DP and the director to make sure that certain elements were done on blue screen that will help facilitate our compositors and animators. In a sense, this is more of a VFX show, so we approached it like, “What is our typical bread and butter animation pipeline for Titmouse and how do we apply that to this VFX way of thinking?”

Juno Lee: Once the show is shot and edited, that’s when we really begin our portion, which is laying it out, making sure that Zorn fits into all the shots.

DS: Titmouse is used to being in control of the productions, but this one is totally different.

JL: When we’re working on animated shows normally, we, the directors, are in charge of everything. We get the show, we board it out and if something’s not working, we talk to the writers, and change it. If the board artist turns out a squeaky angle that’s going to be really tough to animate, we change it. On this one, it’s shot the way it’s shot. When we get the footage, part of the challenge is to figure out how this is actually going to get executed. Little things like how, in live-action, jump cuts and 180 breaks aren’t that big of a deal, for some reason your brain doesn’t really mind them. But when you’re looking at animation, it pops.

JR: It’s a weird hybrid VFX-animation-live action show, but we don’t really get to benefit from the tools that those individual genres have at their disposal. In a typical VFX show they previs every shot beforehand. We, unfortunately, don’t have the time to do that. We are, for lack of a better word, in a reactionary as opposed to a proactive mode, with regards to the composition of Zorn in each shot.

DS: Talk me through the process once you receive the script?

BJK: Lord-Miller and Fox have been awesome about really keeping us collaborating with the writers. So we get a script and we figure out, “Can we do this? If you want this moment to be really crazy, we can do this instead. Make this moment more subtle so we can devote more time to making Zorn do something even more insane.”

JR: We have production and concept meetings at the launch of every script, prior to shooting. Barry and I ended up having to be the stopgap on what we could really achieve with Zorn. You want a writer to be able to write a funny joke, but the execution of that joke affects us in very different ways. Unfortunately, our schedule and our budget doesn’t allow for us to achieve all the things that they want.

We started categorizing different types of shots. If Zorn is doing nothing but talking to an actor, that’s a standard shot. If he was a picking up a live-action prop, like his office phone, that’s one type of shot. If a live-action actor is picking up Zorn’s sword, that’s a very different type of shot because animation still needs to be done, but it doesn’t necessarily involve Zorn. Sometimes it involves a CG-animated prop that we have to then model and rotoscope. Another type of shot would be 100% CG, if it was a CG prop, for example, that flies through screens. Then we had this term, “major events” -- the foundational prop, gag or story beat in this episode – which I would use as sort of currency with the writers and the producers. I’d say, “If this is what you want the most attention paid to, then we’ve got to cut out 10 or 20%,” or maybe we lose some of these other incidental types of shots.”

JL: It’s important to point out that it’s not necessarily done this way to make it easier. We’re always trying to guarantee the best product possible. We understand what is going to look good, what will work and what won’t. The writers are not really thinking about the logistics behind any of that. When we suggest certain ideas it’s because I know that if we do it exactly the way it’s written, we’ll probably get it done, but I can guarantee it won’t look great, because of the time and budget constraints.

DS: Is any storyboarding done on the live-action side? What do you have to work with from the time you’re presented with a script to the time that you’re, as you say, in reactionary mode?

BJK: The actor helps a lot for those standard Zorn scenes. Sometimes I would go with them to location scout and I would take my own photos of the angles that I think would work best and start storyboarding really rough thumbnails, just enough to communicate to all the departments what we’re going to shoot. Then I’d send it to the directors and the writers to see if I’m doing it right or not.

JR: It was a very, very rough previs sort of thing. I think it ended up being a relief for a lot of the directors to have even just the roughest idea. Quite honestly, they don’t know what can be achieved on our end. They would often happily take the previs boards that Barry did at 1 am the night before the shoot. If it works for them then it’s a win-win.

DS:  Talk me through the process once you get the live-action footage. What work are you doing for the rest of the episode?

JL: There’s an initial edit, so we have to do what we call a storyboard pass, though it’s a hybrid layout-board pass really. We break it up into sequences on Adobe Animate and then we place Zorn in, give him the roughest approximate expressions and a little bit of acting, so that when people watch the edit, they’re not just staring at blank plates with people talking to nobody. We populate the edit and then that goes back to Fox and notes ensue, changes are made. Once the animatics are locked, that’s when we break it up into individual shots.            

DS: Is that using your own editors, or is this with an overall show editor?

JL: We kind of have two separate edits going. One is the creative folks at Fox, the directors and the show runner, Sally, Eric, and the editors over there. We have to keep our edit conformed to theirs. It’s like two living edits. For example, as we’re doing something, we need to be able to cut it in and see if it’s working. Even to deliver to them every iteration of the animatic, we have to be in sync with each other.

DS: That sounds kind of challenging.

BJK: Gnarly is the word.

DS: When do you cut in the audio?

JR: That’s a variable that makes things complicated. Our process relies heavily on Jason Sudeikis’ audio. Sometimes there is temp audio to provide some sort of pacing, but then Jason’s inflections and his improv can completely change the whole feeling of a shot. We have to plan out the degree of finish that we would put on every drawing because so much gets thrown away or is rendered useless when Jason’s audio is actually put in.

DS: So you do a blocking pass and then continue from there. Talk a little bit about the animation process after that.

JL: Once the animatic is locked, all the scenes are cut out into their individual shots, and then we start the layout process, which is much tighter than the boards. Then that goes into animation.

One directive was to try to make it look not like Flash animation whenever possible, which robbed us of all of our typical tools that help you tween or copy, paste and reuse. Lord-Miller pinged right away when something was a little bit too flashy and digital. More than any of our other shows, we were actually drawing every frame. I refer to it as trad-digital, because we were mimicking a traditional pipeline at almost every step, from boards to character layout to rough animation to keys to inbetweens to color to comp.

BJK: When we did the pilot presentation, Zorn was more of a puppet-style animation, which is not that far from Space Ghost and some old Hanna Barbera characters. Drawing every frame just kind of adds to that weird inconsistency a lot of those old cartoons had, though Zorn still looks way more defined than a lot of them. It forces us to have to do things in a new way that isn’t conventional, which is the point of the show.

JL: If you look at the old Filmation stuff, it’s a lot of stock re-use. We can’t really use stock because every shot is different. There might be slight differences in perspective, the way the camera captures the field, so we can’t just re-use a lot of the assets. Almost every layout that you see is drawn from scratch. It’s the traditional way. A lot of people compare and describe the show as like He-Man, but it’s really not. He-Man is in one angle the whole time.

DS: What was your approach to integrating such a cartoon-like figure into a live-action setting?

BJK: We’re doing 3D style compositing and VFX but with a 2D drawing. We get to use a lot of the new tools for VFX compositing, like with camera tracking, so we can have a handheld shot with Zorn walking around and we can make him feel like he’s really there. We’ve just got to be careful not to make him look too flat or papery. That’s where the good drawings come in. The compositors are really creative at integrating him with shadows, and he has his own sense of lighting. His color will always pop a little bit more than the scene, so he always feels like he’s still a cartoon. We found when we tried to make him integrate too much, he’d end up looking too dark or noisy.

DS: What are the other major challenges in executing the show?

BJK: It’s the small things. Zorn doing anything slow, small, and subtle that interacts with a real live-action object.

JL: Yeah, big action, crazy stuff, that’s fine. But him slowly pouring coffee into a live-action coffee mug -- those are the things that are the hardest to pull off convincingly.

JR: They’re filming in 23.98 and animation is mostly done on twos at 24-frames-per-second. You get this discrepancy in frame rate. When Zorn does something really slow, that’s essentially done on ones, so we have to roto that movement and it’s almost like the Uncanny Valley where it looks too real. Lord-Miller in particular thought some of those shots looked like CG, and that we lost something of that traditional animated look.

Also, I think no one, including us, really knew what this show would need in terms of caliber of artists, the number of artists, or the amount of needed time. Then there are limitations that are inherently part of working with certain pieces of software, and also just our own internal level of standard of excellence that we always try to deliver. Trying to juggle all that. Juno calls it death by bee stings -- little tweaks and adjustments to make sure that it’s convincing and also achieves all these other mandates.

Chris is a writer & producer based in Shanghai. He’s the founder of the China Animation & Game Network, encouraging communication in the industry via live creative networking events.