'Eli Stone': Pushing the Craft of Immersive Greenscreen

Whether it's a scene taking place on the snow-covered peaks of the Himalayas or an artillery barraged WWII battlefield, all it takes is a few brief minutes to get clued in to the fact that ABC's new fantasy-tinged drama Eli Stone is not your typical legal show. It stars Brit actor Jonny Lee Miller as the eponymous San Francisco legal eagle who's made a life stomping on the little guy by making millions for the corrupt. But his morals-free high life hits the skids when he suddenly starts having hallucinations of everything from a dragon attacking San Francisco to a bi-plane dive-bombing him on the city streets.

In the pilot, Stone discovers he has an inoperable brain aneurysm that's provoking his wacky visions, including semi-regular musical visits from '80s pop legend George Michael. Together, Michael's songs and the mysterious vision of the week point him to a down and out client that really needs a legal hero. Bucking his partner's confusion, Stone takes up the fight of that little guy and each week some amazing visual effect spurs him onto the right path. It's definitely weird, but also charming and most definitely the kind of vfx show you would never expect to see on weekly television without a super-sized budget and months to produce.

But it's not only Stone performing miracles every week. Stargate Digital of South Pasadena and Vancouver has been granted the task of bringing to life every bizarre destination and hallucination that comes out of Eli's addled head and that's no easy feat. Matter of fact, a show of this visual scope would have been impossible to produce weekly only a few short years ago, but in that time Sam Nicholson, CEO & founder of Stargate Digital, has created a vfx shop that prides itself in providing resources to filmmakers that need to have the impossible become possible.

Stargate had worked on a long list of ABC Studios shows, including Grey's Anatomy and Ugly Betty, before Eli Stone came to them in 2007. But it was their virtual back lot technique and greenscreen technology used on those shows that made them the perfect fit for Eli Stone. "It's was a very challenging pilot," Nicholson admits about the show's concept. "In particular, Eli goes to the Himalayas and then he has all these fantasies about being in San Francisco in his childhood. And then the series goes even farther, with dragons and warfare and more. It became very apparent that the show would not be able to produce that in live action. I don't think television budgets are allowing groups to go the Himalayas," he chuckles. "So our virtual back lot process became a necessity to realize Eli's dreams."

First off, everything in the show is massive visually, from Stone's glass encased legal building to his exotic hallucination destinations. Show creators Greg Berlanti and Marc Guggenheim were writing ambitious scripts with average budgets and the only way to pull it off weekly was for Stargate to make their green screen process a part of all the production shooting. "Generally, we do what I call "sequence budgeting," Nicholson explains. "We take a look at a five-page sequence in the Himalayas and say we think it will cost X or Y. We set it up so the director can block it and shoot it anyway they want; it's very free form shooting, as if you are there. Then we put it all together at a flat price. It adds a level of cost control to the production. We work from pre-production all the way through delivery, which is generally about 24 hours before the air date. We design it to ask what needs to be real and what can be virtual, and that's where you save money. You don't over build and don't go to places you don't need to go. We say if we can do this in a parking lot you will save a lot of money."

Nicholson says, "For the Himalayas [in the pilot], you get a few snow blankets and rocks with a bunch of greenscreen and you can shoot six or seven pages a day. We built the Himalayas out of high-res still images and then shot everything in the Disney parking lot. We were doing immersive 360 greenscreen to create all the different environments and ended up creating a template and a style that could be carried into the series from the pilot."

In the past few years with their immersive greenscreen, Stargate has gained a great industry reputation for being able to expand the production horizons of set-locked productions. Nicholson explains, "We directly developed [the virtual back lot] for about seven to eight years. Greenscreen or bluescreen shooting is nothing new but what is new about the way we approach it is to have these completely immersive environments that are 360 degrees in extreme high-resolution: 20 to 30 thousand line resolution. What that gives us is the ability to have complete freedom of camera for the DP and the director so they can shoot it Steadicam, off cranes or hand-held with any lenses or any focal depth. We have systems that can preview the composite in HD in realtime with pan, tilts, zoom and focus. It gives us six axis of compositing capability in realtime. So now when you are standing there, you can say let's raise up the camera six inches because now we can see the vanishing point on the horizon when before you couldn't see that. Now the actors and directors and editor, particularly, get to walk away with a pre-comp that they can start cutting to."

It not only frees up the creative choices, but the practical improvements have also shaved off budget expenses. "We have special greenscreens that we have designed to cut the cost of the rigging," Nicholson explains. "The rigging can be cost prohibitive in itself. For a 100-foot wall of greenscreen that is 20 feet high, generally, in traditional methods, costs $50,000 to rig. We have cut it down to about $5,000. Our greenscreens roll so four grips can roll a 100-foot wall of green around!

"And the key to conquering a television schedule is having a library," he continues. "I parallel this to visual mixing. If you go into a sound effects facility and you can't pull down sound effects, you can't get anything done. You may have to do a little bit of Foley and creation of special things, but in general 90% is pulled down from a library. What we have done on a visual basis, which is why this has taken years to put together, is we have assembled a two to three thousand-hour library of HD backgrounds: London, Moscow, Tokyo, New York, Washington D.C., San Francisco… the Himalayas," he laughs. "Those become digital sets. It's like a digital IntroVision basically. But what happens when you get into digital space, is that you can break out and do complete freeform camera moves in 360 degrees."

Nicholson says that their team has gone all around the world to capture environments for their library. "We shoot circle vision with 8, 10, 12 HD cameras at once and then we use computer simulations to in-between it. The computers help with filling the gaps. And then the artists themselves are extremely experienced at environmental green screen compositing. All the elements have to be done correctly to make it work. It has to be lit properly, blocked properly and the background plates have to be shot properly. The director also has to understand at what point he could fall off the edge of the world. And the cameraman has to get the lighting exactly matched to the background. But when you get all that together with really high quality backgrounds, you feel like you are there."

They specifically created Stone's law office by getting footage from the source location. "We went up to San Francisco and shot a lot of material there and then we went to a location and basically digitized the Nikon Building in Irvine, California. We backed [the cast] up with greenscreen and rebuilt the Nikon Building and pre-composited the gigantic office building with all the stuff outside and then comped that into the actors. Now, you can bring the locations to the actors instead of the other way around. For a series, with ultimately photoreal sampled sets, if you will, and virtual set extensions, it's kind of like living translights. It's totally possible and very economical and a ton more controlled than with live action. We even rearranged the buildings in San Francisco a little bit to make sure the TransAmerica would be predominantly featured. And if there is any signage, you take it out. All of it makes for a more controllable environment for directors."

Every script brings the Stargate team a new set of intense challenges that really make or break the show's vfx core conceit concept. Nicholson admits, "It is a very challenging premise because each week is different. It goes anywhere from digital, fire-breathing dragons to flying to London to shoot George Michael on greenscreen and comp him back into the footage because he couldn't make it over [to Los Angeles]. We are basically utilizing vfx to solve production problems. Some are traditional CG animation techniques, some are virtual back lot techniques and then some are just figuring out how do we get George Michael into 10 scenes when he can't come to the shoot? Our vfx supervisor on the project, Peter Vazquez, is very adept as he is a DP in his own right. He understands both the on set work intimately and the lighting of green screens and the compositing. He gives us a very seamless interface with the principle photography team so that when you are talking about a battle scene, it is lit and shot properly. We always stress shooting the material perfectly if you can. No artist in vfx wants to deal with problems that are just passed down from principle photography like bad greenscreen or underexposed work that is grainy, so we go to great lengths. And the production has been great about getting the best possible quality out of the green screen so when we put it together it's a one touch key if possible. In general with the volume of TV, you have to be more careful that the material that comes is almost flawless, otherwise your artists will revolt!"

To help the compositing flow, Nicholson says Stargate has developed a proprietary asset management system called ADAM (Advanced Digital Asset Management System). "What that does is link all of our artists and the shots in all the shops on one single chat room. Everyone can review all the materials and it's connected to all render farms in both L.A. and Canada so we can see renders from Canadian artists before they do in composites and in 3D. We do simultaneous rendering in both countries at once. We have a lot of storage with almost 200 terabytes. We synch it together with an Avid DS at the core of the editorial systems in both locations. We are predominantly Maya and a little LightWave for 3D and then we are all After Effects based because it's networkable. We are not Linux based. We are Windows based which allows us to write a lot of proprietary code fast and use off the shelf software packages very quickly. We are shooting with the latest digital technology for the greenscreen, generally the D21 or the F23 or lately the F35, which is a brilliant system. You put it all together and you improve every part of the system from previs to design, all the way through delivery."

One of the biggest pluses about working on Eli for Stargate is the variety of what is asked of them, such as creating a fully modeled and composited dragon. "The animators love doing creatures," Nicholson smiles. "I don't know a single 3D guy that doesn't jump at creating a character. They love it and we have some very good creature animators. They really got into it and pushed it over the top. Tony Ocampo, our senior modeler, also did the Golden Gate Bridge collapsing sequence and it looks unbelievable. It is perfect. Basically with this show, you have the idea of feature quality effects on a television budget and schedule. I wouldn't say they are as polished, you don't do 70 turnarounds and polish it and polish it like a feature. If you look at it under a fine microscope, sure there are things we'd like to continue working on, but a feature shot is generally done at about 90% then the last three months is tweaking. We do that type of work well and our artists do that type of work too, but if you can get the work right at 90 to 95% within 10 days, then you are doing really well on TV and we are even working at full feature resolution, 2K and well beyond HD!"

As to shot delivery numbers, Nicholson says, "We average about 200 shots per episode and they are complex, whether it's a fire-breathing dragon fighting Eli on horseback or a battle scene with spit fires flying through and bombs going off in one shot. It's a challenging show but we have streamlined it in a creative sense and there is a great communication with all the people on the show. We do have a dedicated team but we have the ability to add to it dramatically if we need to with our facility in Vancouver. We can lean on them to add if things are tough. We focus and refocus teams rapidly, so if there is a bottleneck for any reason, we can put 50 artists on it and push it through. I think it's the advantage of a facility like us. The point of a place like Stargate is that we can bring a great amount of horsepower to bear quickly and refocus.

"Then it really becomes a creative partnership with the show, that they have to trust you when a shot is delivered with only one or two opportunities to change it," Nicholson continues. "They know they can't tweak it endlessly or we won't make the deadline so there is much more creative responsibility that lands on a digital artist on television because what you deliver is probably what will go on the air. It's not a matter of if the client likes it or not but more of… is it going to make it? You can't have a green hole in the show," he laughs. "When you are doing 200-300 shots on a weekly basis for a show like Eli, there isn't time so you have to have an understanding with the artists that they need to take responsibility for themselves to know when it's airable and when it's as good as it can be. They almost have to read the minds of the EP and the director and that's where the vfx supervisor comes in critically because nine times out of 10, the vfx sup is approving shots. If it isn't what was talked about in a production meeting and isn't in creative agreement with the director, then there is eventually going to be trouble. So the relationship we have with Mark and all the writers is so important. On a feature, you can afford to be relatively compartmentalized because there is enough time. What's strange to me is that many times the supervisor, when a feature gets broken up over many houses, will never meet the director. There's a chain of people that don't meet! What we try to do is make that line as short as possible, with maybe one or two people. The director is always invited to sit with the artists and, if not, the artists go to set and there's only one supervisor so it's a direct link. The message doesn't get translated too many times and you are capable of hitting the needs. It's very interesting with all the different shows that we do, we really try to get a mind meld between the supervisor and the style of the director. We have supervisors that are DPs, or strong in 3D or 2D or came out of producing. Each one of them will go after a job in a particular way and what we want is to find a complementary match with the skills set that is on set and we have that with Eli Stone."

Tara Bennett is an East coast-based writer whose articles have appeared in publications such as SCI FI Magazine, SFX and Lost Magazine. She is the author of the books 300: The Art of the Film and 24: The Official Companion Guide: Seasons 1-6.