Pixar Goes 3-D

Today's release of Up marks Pixar's first stereoscopic 3-D feature, so we chatted with Josh Hollander (director of stereoscopic production) and Bob Whitehill (stereoscopic supervisor) to learn about the studio's philosophy behind the digital revitalization of the format that's taking animation by storm, including a sneak peek at Toy Story, Toy Story 2 and Toy Story 3 in Disney Digital 3-D. The first two landmark Pixar films will play on a double-bill for two weeks beginning Oct. 2, while Toy Story 3 bows June 18, 2010.

Bill Desowitz: It's clear in viewing Up in 3-D that Pixar's approach is dynamic but unobtrusive.

Josh Hollander: Yes, the most gratifying thing is that we put a lot of time and energy and thought in our approach to 3-D and we never really knew if people would get it. To see some of the online reviews coming out of Cannes, it seems that people really got our nuanced, subtle approach. We want to pull people into the movie and not push them away from it with a lot of gimmicky stuff coming at ya.

We wanted to really understand this new tool before we integrated it. We did a lot of tests with WALL•E material, Ratatouille material; working on the Toy Story re-releases. We started with Up as soon as images were available. With that variety of imagery, we started pushing the limits of what's comfortable and what's not in all aspects, including cutting from shot to shot, continuity, far away in z-space forcing your eyes to go more wall eyed, vs. objects coming out into the audience in z- space, forcing you to become more cross-eyed. We had a core group of people on this: John Lasseter, Pete Docter, [Up Producer] Jonas Rivera and [Up Layout Supervisor] Patrick Lin. We looked at all this material and viewed the latest films in 3-D and put it all into the overall Pixar aesthetic.

BD: And how would you describe that aesthetic?

JH: We came up with our general approach, which sees 3-D as just another tool in a filmmaker's toolbox to help tell the story. So we believe we've found an approach that works. In calm moments, in sad moments, when Carl is alone, we really compress that 3-D depth to make you feel closed in and small. And, of course, with all the topography of the Tepui as the movie progresses, and the high notes and big emotions and action, we push that depth out to support those story points. To use the metaphor, you want to bring people into your story, so if you apply a literal approach, you want to bring people into the movie screen. We don't want them walking out saying that it's great 3-D.

BD: What were some of the early lessons you learned from the test footage?

JH: What was really interesting was, not knowing how 3-D was going to work with editing and with composition, how good camera and staging play really nicely in 3-D. And this turned into a motto for us. We did about eight minutes of Ratatouille as a test and it was really stunning. We selected moments from across the movie with foreground and objects far away and fast cuts and slow cuts and pans and vertically and horizontally dark and light shots. With that said if you have an extreme foreground object but the focal point of the shot is farther back, it can be jarring, so we keep that in mind. We learned how to use the floating windows and other aspects of 3-D.

BD: You implement your 3-D planning as early as possible?

JH: Yes, we start in layout and figure out how to work 3-D into that. There's a chart to map out the story arc of the film and how the layout composition works and what Bob did was work with Pete and Patrick and mirrored a 3-D depth chart with their chart. We move the stuff over into 3-D layout, work out the shots and send it off to review. With each project, we might find subtle differences in how we want to use 3-D. The obvious anomaly is Toy Story and Toy Story 2, where we don't have the luxury of planning our use of 3-D.

Bob Whitehill: We were fortunate to fall into Up where the lens palette is mainly normal so we don't have to battle long lenses. Those will collapse the characters and draw the background in. And we don't have a lot of abnormally wide lenses, which can exaggerate the depth of field and double up what you need to do in 3-D. In Up, we have these wonderful vistas and a zeppelin fight that lend themselves so well to 3-D. We uncovered a gem for our first one out of the gate. Up required little manipulation.

One of my favorite moments is when Carl is looking at the Adventure book and [learns the movie's great lesson about life]. The general idea is that when Carl's life is flat and contained, the 3-D would be the same. And that when he is with Ellie, the experiences become more dimensional and deeper, so what I did over those shots in the book, was take each one a little bit deeper as we cut back to Carl, so that by the end, he looks more fully rounded and more dimensional -- more human, if you will, by reacting to what he sees in the book.

BD: So you've come up with a 3-D playbook?

BW: We established a trade template where we wound up putting the point of interest where your eyes are drawn at the screen depth, and then pulling the foreground forward and the background back around that point. Once we found our creative footing on that front, then pipeline wise, we were able to dive into our proprietary tools and set up a system that is pretty straight forward to add in the right-eye camera and to use the existing camera as the left eye.

BD: What were the challenges of converting the first two Toy Story movies?

JH: What we've done there is Bob, John Lasseter and I (along with a few others) have mapped out how to use 3-D in support of the story. We're all so familiar with those movies that we wanted to stay out of the way of the stories and really augment them… and set up Toy Story 3.

BW: The original Toy Story, of course, was the first to be done in computer graphics so they were already very conscious of using the z-axis and using the camera to push through scenes because this was the first time that you were able to do that and have characters inexpensively shaded and lit from every angle. So we discovered that there's a lot of really great 3-D that comes along with that idea. In the moving van chase, there's shot after shot of movement across the z-axis, to and from camera, to and from camera, which works incredibly well in 3-D, and is a byproduct of the fact that they had this new technology that they really wanted to use. Starting in Andy's room, we were able to keep things spatially consistent with the 3-D and in proper scale, so to speak, to what the toys experience in that room. But when they went into the human universe, we were able to really blow things out and make things deeper. It was interesting to find that what we were doing with 3-D fell in line story wise and style wise in terms of changing the world to mirror the toy experience. Pete Docter has always said that we made our films in 3-D but were never able to project them in 3-D and now we can.

JH: I was giving a talk about 3-D recently and asked for a show of hands for those who've tried to open up a Microsoft Word file from five years ago. A few hands went up. From 15 years ago? Everybody gets the issue of backwards compatibility with new technology, so our first and maybe greatest challenge was gaining access to 15-year-old files and figuring out how to port them forward to run on modern technologies, and we were fortunate to have some original technical artists who worked on Toy Story to assist us. Once we conquered that, we were able to open up the shots using our modern stereo camera rig. From there, the challenge is how to set stereo depth in these shots in such a way that's pleasing. The next challenge is get them to render clean and successfully and matching the original frames. We use the term bit rot in general to refer to digital assets decaying over time. As Toy Story was being made, there were tons of fixes and adjustments and enhancements, so, by the end of the film, you might not have been able to render a shot from the beginning of the process and have it look the same without some work. We are tweaking the shots to make them look the same as the original output. Our goal has been to make this imperceptively different.

BD: What's going to really stand out in 3-D?

JH: I think there are certain shots and sequences in each movie that are going to be phenomenal in 3-D. We've applied the same general aesthetic. You can picture the opening of Toy Story 2, the video sequence with Buzz going into Zurg's lair and that's going to be amazing. We've been reviewing Toy Story a lot recently and the gas station sequence where Buzz and Woody are on the verge of getting lost and going to Pizza Planet. The scale of that environment plays well in 3-D. Another thing that we're learning is that natural depth really supports the 3-D experience. And there's a bit of a challenge being in an enclosed space, such as Andy's room, where you want to refrain from going over the top but make you feel the 3-D.

BW: As we move forward and think more and more about 3-D and the process of our filmmaking, that's where we have yet to discover where we'll take it. We're compiling a list of things that might've been done differently or enhance the depth of a shot or the 3-D experience of shots in the future. It's up to each individual director as we move forward to decide how much they want to fold 3-D into the creative process. We'll be there with our tools and our tricks and knowledge.

BD: Where are you at the moment with Toy Story 3?

BW: Right now they are concentrating on the general experience of the film. As part of a presentation that I gave them, I provided a lot of examples from Toy Story and Toy Story 2 and what we've learned. What can we do to make the 3-D version better without adversely affecting the mono version?

JH: Toy Story 3 is a lot of fun having come off the first two and getting our hands into the new story. There are a lot of similar environments and sets and it feels like we're primed for TS3. The information we've gathered so far from all of our early experiences is going to dovetail nicely.

Bill Desowitz is senior editor of AWN and VFXWorld.

Bill Desowitz, former editor of VFXWorld, is currently the Crafts Editor of IndieWire.