If I ever need to really “understand” something related to visualization or virtual production, I talk to Ron Frankel. There are certain people I’ve met during my tenure at AWN, people who have a knack, a gift if you will, of being able to explain often complex ideas in a succinct and understandable manner. And make it interesting. Ron, I dare say, could probably describe the inner workings of a cyclotron in a way that would actually make sense, in a way that would make anyone suddenly feel a career in Physics was now possible. Even desirable! Alas, I am neither succinct nor understandable. But after a conversation with Ron, I always feel a wee bit smarter.
As the founder and creative director of Los Angeles-based Proof Inc., Ron has been a seminal figure in the ever-expanding feature film visualization business for more than a decade. With a wealth of experience leading teams on some of the biggest, most effects-laden features, Ron understands first-hand the value that collaborative storytelling, design and visual development efforts can bring to a film. Over the course of several interviews in three different countries, Ron spoke to me at length about the art, technology and business of feature film visualization, as well as the dynamics, the politics and the growing awareness of its value within the filmmaking world.
Dan Sarto: Before we dig in here, help me understand the relationship between virtual production and previsualization.
Ron Frankel: Sure. The topic of virtual production, obviously, is a big topic these days. There are two ways people think about and approach virtual production right now. One of them involves the changes that are happening onset and nearset. So, virtual camera, SimulCam, real-time computer graphics onset, that whole thrust of these new technologies. Then, there is the broader definition of virtual production, which starts to encompass all of the digital technologies and visualization technologies that are happening from the early days of pre-production all the way through post. Three, four, five years ago, what we were talking about was the digital evolution of digital production, which was the migration of digital tools into pre-production as part of the design and planning process for feature filmmaking. Believe it or not, that’s only happened in the last few years. It’s just really been within the last 10 years that you’ve had 3D set designers working in art departments. Up until then, it was draftsmen working with pencil and paper, whereas in the visual effects world, we’ve had 20 years of really robust digital effects.
In the pre-production world, it’s really just been in the last 10 years that those technologies have come into play. And previs was part of that. Previs was part of this migration of post-production tools into pre-production. Back then, when it was happening, we referred to that as digital production. Now, with the advent of real-time onset computer graphics, like the stuff that you saw in Avatar or Real Steel, like the Lightcraft Previzion system, now we are talking about virtual production. Regardless of what you call it, what is happening is kind of fantastic, because there really is starting to be cross-departmental, cross-production collaboration. You’re beginning to see how all of these tools are beginning to transform the creative process. For me, that’s really the key.
When I got involved in the world of previsualization, what really drove me, what motivated me, was this idea of creative decision-making. Creative decisions are very difficult to make alone. Creative decisions are incredibly difficult to make in groups, trying to get a whole group of people on to the same page with the same creative idea. So, one of the things that’s really changed for me over the last year or two is, for instance, is that with my company Proof, I now refer to us as a visualization company. We certainly do previs, that’s a part of what we do, but it really is only a small aspect of what we are engaged in now. In addition to doing previs, which is “pre,” we are doing postvis which is “post.” We are doing onset visualization, which is the stuff that’s happening during principal photography. So, it’s not really accurate anymore to describe us as a previs company, because that ignores essentially two-thirds of what we are engaged in. I think the business is still predominantly about previs, but those other parts are growing at a very encouraging rate right now.
DS: The last few months have seen even more turmoil in the visual effects community than usual. How does the continued shakeout impact your work and the studios you work with? Is there more opportunity?
RF: My business experience with Proof has been that previs is being relied on even more heavily. Filmmakers are taking the time earlier in pre-production and production to make creative decisions rather than waiting until post. We’re being use more as creative collaborators on a film. Something definitely needs to be changed in the way the visual effects industry works. What I’ve been seeing is that productions are waiting longer and longer before awarding the visual effects. There has to be more creative collaboration with visual effects earlier on. I’ve always enjoyed opportunities when the previs team can actually work with the visual effects vendors. Those end up being the best collaborations because we can take the issues and concerns of the visual effects companies and reflect those to the filmmakers. The whole process can benefit from the integration between pre-production and the post-production vendors, breaking down those barriers between the two and not just relegating post to post. I don’t know where that fits into the business model but on a purely creative level, everyone would be a lot more satisfied. You’d end up with a happier industry in general.
DS: Wouldn’t that be nice?
DS: The recent Motion Picture Academy VFX Convergence event on visualization in storytelling was extremely insightful. Really, a fantastic evening. What’s the key message people hopefully came away with regarding feature film visualization?
RF: That’s a great question. I hadn’t thought about that [laughs]. It would be great if people came away with a better understanding essentially of how prevalent previs is throughout the film production process. It still feels to me often times that people think it’s a “luxury good,” a specialty item that is reserved only for the very top shelf big budget productions. The truth is that yes, those films do get a lot of attention. They get all the press because they are very exciting. But, you’re really finding previs, postvis, the whole spectrum of visualization being utilized on so many more films than you would ever expect. So, hopefully people came away with an understanding that this is a practice, it’s a discipline that has its uses on a wide variety of films for a wide variety of reasons. It’s not just about the big, vfx-driven feature hero genre film.
DS: You’re not just a front end process for visual effects.
DS: Are you seeing any significant changes in the acceptance and use of previs in Hollywood?
RF: Sure. It’s been expanding. It’s more widely accepted. It’s more widely used. Previs is being used for an even broader range of applications than I would have imagined just a few years ago. Previs on lower budget, mid-range, almost non-visual effects films is pretty frequent. Previs for the big visual effects tent-pole films, we’re almost getting to the point where we’re prevising the entire movie. With the exception of the handful of dialogue scenes, we’re doing all the major action beats, which for these big superhero films, is basically the lion’s share of the film. We’re even getting involved in areas where we’re testing ideas for marketing, prevising trailer ideas and trailer shots so that the marketing department can test those to see how they’re working before they go film them. We’re getting involved in areas of the film production process that are actually outside of production.
DS: Why would a studio not want to previs every single thing they possibly could? Why would anyone want to step onto a set without having at least gone through the process of seeing what they should expect to encounter during filming?
RF: For complex filmmaking, for visual effects filmmaking, where walking onto a set is not just that organic process where you can just “wing it,” because everything is being done by different people, in different places at different times, and your live action component might just be your actor, for those instances, you have to previs it beforehand. Or else, you have no idea what you’re doing. You have no idea the scope, the scale, the pacing.
We just finished doing some work for the film 42, the Jackie Robinson story. That was really just a location shoot. There was a bunch of previs that went into planning what they were going to see regarding the CG set extensions. So, in an instance like that, it made sense not to previs down to the specific shot, not to give them a blueprint of “this is what your shot’s going to look like.” It was more about giving them an envelope to work within, saying, “Your wide shots will be about ‘this’ wide. Your tight shots will be about ‘this’ tight. As long as you’re working within this envelope, you can do whatever you want. But once you get outside that envelope, you’re going to break the visual effects budget.” That was their big concern - how many wide shots and how wide could the shots really be. They didn’t have a really big vfx budget.
For 42, Proof was initially brought in by Hammerhead, the VFX vendor for the film, to design some of the hero moments. They ended up working closely with Director Brian Helgeland, Production Designer Richard Hoover and Visual Effects Supervisor Jamie Dixon to devise strategies that maximized the visual impact of the shots in the historic stadiums, without breaking the bank on the visual effects budget. The previs was used to determine how many extras were required for specific shots and where digital set extensions would start and end. 42 images courtesy of Warner Bros. All rights reserved.
So, what you’re seeing is there’s really no instance where you wouldn’t want to walk onset with some level of previs design planning about what goes on. The question really is, to what detail? Can you go through with broad strokes? Can you just give a general envelope, a general sense of what the production is going to encounter when they get onset? Or, do you need to get down to the level of specificity of, “Use this camera with this lens at this height and this is the shot” because there are so many other complexities that come into it?
DS: So why would a director not want to use previs?
RF: Amongst some directors, I think there is a sense that using previs is going to force them into making too many decisions too quickly, and that they will then be held to those decisions. It’s hard when you get into previs. There might be decisions about the environment. There might be decisions about the characters and how they look. There might be decisions about the cinematography, the pacing, the action, the beats. You have to respect there is a process that has to go on. That process doesn’t start by “making” those decisions, that process starts by methodically moving through all the steps that you need to move through in order to “make” the decisions, which is typically what happens on stage. So, I do sympathize to some degree with that sense that sometimes, previs puts the cart before the horse and forces a lot of decisions to be made when the production simply hasn’t had time yet to consider them, to deliberate over them and to decide on them.
Sometimes you see things creeping into the previs, and suddenly, they are accepted, and suddenly they are “there.” Suddenly, they show up in the movie and someone says, “Well, where did that come from?” and, “I know it was in the previs. So, therefore, it must be, because it’s there!” So, I can sympathize, because often times, the process, depending on how the previs is managed, can get shortcut and short-circuited a little bit. I don’t think it has to happen that way. And I certainly think that there is a way in which you can essentially respect the creative process, so that the previs is an accurate reflection of the decisions that are being made “as” they are being made. Previs can become a great medium through which decisions can be evaluated and then made. But as I said, you do run the risk of the cart getting in front of the horse at times.
DS: I can see, for instance, a studio looking at previs and telling a director, “That’s great! That’s the film we want you to make.” I can then see a director saying, “Well, that’s really not how I make films.” Can you see a situation coming where a studio will say, “Unless we can see what this film is really going to look like, we won’t finance it.”
RF: Yeah. There is certainly a trend out there. It’s not very prevalent, but it’s out there. It’s kind of the dark cloud on the horizon, where previs is being used almost as a contractual document from the studio, where the studios are insisting that the directors previs a sequence, and then when it goes into production, are basically holding that director accountable to delivering that sequence. Not as an idea of the sequence, not as the germ of that sequence. They are saying, “No, that wasn’t the shot that’s in the previs.” Now, that’s a little bit of hyperbole and exaggeration, but like I say, there is kind of the feeling that’s the dark cloud on the horizon. There are instances that I have been involved in where the studio has pointed back at the previs and said, “No, no, no, that’s what we agreed to you making, not this other thing.” So, yeah, I think there is a risk.
Real Steel. Images courtesy of Digital Domain. All rights reserved.
There is a broader divergence going on between what’s happening prior to production and what’s happening in production. You are seeing a lot of pitchvis that is being used in pitching the film, in the interest of getting it greenlit, in the interest of getting talent involved, in the interest of getting a studio to bite and to put money behind it. All totally valid and fantastic. I mean, it’s a sales tool and any sales tool that helps sell the product, fantastic.
The risk is when that starts to drift into production. At that point, the filmmaker gets involved. The point of hiring a director is because the director’s vision is going to be different from the producer’s vision and the studio’s vision. So that’s where you can run into this potential conflict of interest between what’s happening during that sales pitch mode prior to production, and then what’s happening during pre-production, where now you’ve got the department heads involved, you’ve got previs involved, and you are trying to craft the vision and make a movie. They are very different animals. There is a risk that they become one in the same thing and that’s the point at which you start to see the studio basically dictating a little bit of the creative vision upon the production.
DS: A lot of finished previs has an unfinished look. In some instances, I can see how that might not illicit a good response from an executive or director who looks at it and is not particularly impressed. How do you address this issue?
RF: A handful of years ago, in 2007 or 2008, I first started looking into producing a more cartoony, ink and paint toon-shaded style of previs. At the time, it just seemed interesting, kind of fun and there might be some instances in which it would have a really good application. What I’d discovered is that our artists were spending a tremendous amount of time texturing and surfacing all of our assets. In many instances, we were doing that before the art department had ever really weighed in on what these finishes were going to be. So we were laying things out as brick, or tile, orange and red, making all these creative decisions that are really in the domain of the art department. But, because the art department hadn’t gotten there yet, we were running ahead. It became a multi-tiered problem.
One, it was time consuming for our artists because they were having to make all these decisions and apply these textures and work at a much higher level of detail, which just takes time. In the world of previs, time is a really valuable resource. Anything you can do to cut down on the amount of time spent is great. Two, it was causing some friction with the production because the art department hadn’t addressed these issues. These weren’t “their” decisions. So, we had to go back and forth with the art department on what we were doing. Three, it was often just distracting. The director would come in and start commenting on the wall color or the floor color, when those really weren’t the creative issues at hand. What we were really looking at was the blocking, the pacing, the timing, the composition. Instead, we’d get comments on the lighting, the coloring, the finishing and the textures.
We had all these problems converging around this one area. What can we do to basically wipe the board clean and make all of these non-issues? So, we came up with the idea of doing a toon-shaded previs. We first implemented it on The Green Hornet. It was a great opportunity. Michel Gondry had discovered early on, when he looked at our previs reel - and I was really proud of our previs reel, it had all this great looking stuff on it - it really turned him off. He’s the kind of filmmaker who already sees the movie in his head, who knows what it’s going to look like. He doesn’t need someone else telling him the “bad” version of what it “might” look like. So it really shut him down creatively when he saw it. I thought, “OK, this is our opportunity. Let’s go in with this toon-shaded style because he draws. That’s how he communicates.” When he’s trying to get an idea out, he immediately brings out his pen and paper and starts to draw. So I thought, “Great, let’s meet him on his turf.” Let’s put together previs that looks like drawing. That will draw out the creative communication between us and encourage that dialogue and collaboration, as opposed to having him be turned off by the whole process. It worked extremely well.
The process we used on The Green Hornet was a little bit cumbersome because it was a two-step process. We actually animated and then rendered, which for me in the world of previs, is a non-starter. I like everything to be immediate and one-pass so the director can see immediately what they’re going to get. But based on that success, we actually coded a real-time CG effects shader that we continue to develop every day, to do that toon-shaded style in real-time. We recently finished our second production with it and have two more productions right now that are using it.
Our clients are discovering, and what we’re discovering, is that it takes you back to a creative space. When you watch a director sit down with a storyboard artist, it’s all about opening up opportunities. They’re like, “Oh, this can be here, and that can be there and boom! it will be a big explosion.” There’s a lot of gesticulating and lots of metaphor and description for what “could” be. When you look at that very detailed approaching realistic style of previs, you get the opposite reaction. You get a lot of, “Well, is that what it’s going to look like? Is that what it’s going to be in the end?” You end up approaching it from a much more closed-in, critical place and not that more creative, opening up opportunity kind of space.
Our clients really respond to it. They respond to it the same way they respond to storyboards and drawings, where it becomes a platform for imagination, a platform for riffing on and sparking new ideas. It has the added benefit of not stepping on the production designer’s toes, because there’s no question that these are not the final finishes, the final textures, the look of the environments, the sets and the props. Interestingly, it leaves a lot of room for the cinematographer. Often it comes up, “What is the role of the cinematographer in previs?” Unfortunately, the answer is there isn’t much of one. They’re not on early enough. Usually, we’re pretty much done with our work by the time the cinematographer gets involved. What they inherit is basically the scene that is pretty well fleshed out and designed. When you go to the toon-shaded style, what the DP inherits is really a blocking pass. A pure layout pass. Sure, it has lenses, composition and timing, but it doesn’t have any lighting in it. There is a lot more room for the DP to step in and build on that as opposed to fighting against that. So, we’re finding it very successful.
For the upcoming Captain America: The Winter Soldier, Marvel requested that Proof devise a visual style for the previs that prioritized motion and composition over color and texture. They used their proprietary cgfx "toon" shader, which they've been developing over the past two years, to give the previs a more illustrated feel. Captain America: The Winter Soldier images courtesy of Disney/Marvel. All rights reserved.
DS: Inherent in the work you do, directors get excited about the way their film is beginning to take shape. However, your design work often has elements that could never be put into the film. Elements of scale, for example, that are not remotely practical. People inherit your work and respond, “There’s no way this can get made.” How do you manage the excitement of early previs against the reality of what can actually get made?
RF: [Laughs] It can’t be done! That’s always a risk. That’s part of the animator’s code of responsibility. For anyone working in previs, just be sensitive to the fact that what we do is a template that someone else is going to inherit. It’s going to become their responsibility to deliver. So, there are a couple of different ways to approach this issue. One could take an approach that says, “I’m going to design this the most exciting way I can see it and it’s someone else’s job to figure out how to do it.” That’s someone else’s responsibility. They’ll figure out how to make it because this is how it “should” be.
That’s not me. The way I approach the work and ask all Proof’s artists to approach the work is from the point of view of saying, “Well, what can we do now to make the next person’s job easier?” What information do they need, how can we set this up so that not only can they achieve what we’re laying out for them, but we can give them that extra bit of information that will help them do it? It’s important to me that everyone understands all the production crew’s roles and responsibilities and how what we’re doing affects their job.
The type of situation we come across all the time is crane shots. The director will say, “Oh, get the camera higher, higher! I want to be way up here!” So, we design the shot as the director would like it. Then, the first thing we do is go to the producer and say, “We just put a camera 120 feet off the ground. Are you guys prepared for that?” The producer will come back and say, “No, no, no. We have an 85 foot crane. That’s all we have.” So we’ll go back to the director and say, “Hey, here’s what the shot looks like from an 85 foot crane. Here’s what we can do to get the same kind of shot, the same feeling, within the constraints of what the production has at hand.” If the director accepts that, great. If not, he has to go have a conversation with his producer to figure out what to do about it. We can facilitate that conversation by being aware of what the constraints are, where and what the realities are. If we go about it blindly without any consideration, then the previs really becomes a hindrance to the process, which is really antithetical to what we’re trying to do.
DS: Besides 42, what else have you been working on?
RF: We just wrapped Fast & Furious 6, which we were on for I believe 18 months. We started in really early-early pre-pre greenlight development, working with the director on some story ideas as the script was being developed. We had a previs team in Los Angeles, working with the director before it moved to London. We moved with the production to London, then stuck with them all the way through editorial. Our postvis team ended up being a final in-house vfx team. So, we were on until the very last days of delivery.
We’re doing two projects for Marvel right now, Captain America: The Winter Soldier and Guardians of the Galaxy. We also have a couple of secret London projects that are in development which hopefully will come to fruition.
DS: Looking ahead over the next five years, where do you see feature film visualization headed? What’s next in terms of technological and onset innovations?
RF: With technology advancing at the rate it is and with productions becoming more and more sophisticated, part of the reason why, and I don’t say this lightly, but part of the reason why I have made this transformation from calling Proof a previs company to a visualization company is because I do think what we do is going to change pretty radically over the next few years. I don’t ascribe to those ultimatums, like there will be no more previs. You hear people saying, “Oh, previs won’t exist in a few years.” That’s probably not true. I think that there will always be a role and a need for artists sitting in a room somewhere working with the director. There might not be as many artists. It might not take as long to do. But, I think there will always be a need for previs artists. There is still going to be a role for them.
The difference is going to be in this blurring of the lines between what happens in pre-production, production, and post. You are going to see a lot more of the kind of virtual production work that you are seeing in productions like Real Steel, where the previs team is now kind of the visualization team. We’re building assets for animating, but we are not doing the cameras. The cameras are being done maybe on a motion capture volume with a virtual camera by the director or the DP. And all that data is being wrapped up and sent off to a post-production facility for rendering. Or from that, we are extracting a camera move to shoot actors on the blue screen. What will happen is a continued blurring of the lines between the roles of pre-production, production, and post in which a previs team is a vis team is a postvis team. They are not necessarily one in the same thing, but there is a lot more fluidity occurring between them.
Photo-realistic real-time rendering, whether or not we ever get there, we’ll see. But the technology we “are” going to see in pre-production, the holy grail of previs, that has always been the game engine. I certainly think that’s going to come up as a real game changer in pre-production. So, rather than seeing teams of 20 artists laboring away doing previs, you might see teams of three or four really skilled animators and a couple of layout artists in there going to a real-time game engine. You are going to see more direct input devices, where you can put a virtual camera in the hands of the director or the cinematographer as they are actually composing the movie. Onset, you will see more of the visualization work where assets that were developed in previs are showing up onset, assets that were developed in previs that we never used for previs.
We actually just finished a job where we did onset visualization for a sequence that we never prevised. We simply created the background. We created the asset and the environment, because the production needed help essentially visualizing where they were in the world. It wasn’t a particularly high-res asset. It certainly wasn’t a photo-realistic asset. But it was an accurate enough asset that the DP could look at it. It was a location that they couldn’t scout. The story behind it was that it was meant to be a practical shoot. And fairly late in the game, at the last minute, for a variety of reasons, it became impractical to shoot there. So, they had to move to a greenscreen. But there had never been a scout of the location. The cinematographer had never really gone there and blocked out the sequence. So, what we did, we brought the location to the director and the filmmaker onset.
So you are seeing more of that kind of work, where the previs was onset visualization. What comes out of that onset work is an HD comp of the action taking place in the background that was recorded by the onset video assist and went straight to editorial. So, now, is it postvis? Well, it was recorded on set. But now they have a temp comp that they can use while the director is editing the sequence together.
You are going to see a continued blurring of all these lines between the traditional pre-production, production, and post-production. You are going to see more continuity and uniformity in the visualization. So, as a visualization company, Proof is essentially trying to position itself as a creative service for clients that can stick with them from these earliest days of development all the way through essentially the final days of delivery. At the same time, you are seeing productions go all international. Production is taking place all over the world. So, we have also established ourselves as an international company with operations in Vancouver, Sydney, London and our home office in Los Angeles. A lot of the shows start in LA, they might go shoot somewhere else and they come back to LA to post, they might stay in Canada or in London to post. So, essentially what we are trying to do is really be a creative resource for our clients at every stage of the production and at every physical place they can go.
Dan Sarto is editor-in-chief and publisher of Animation World Network.