Christophe Hery Talks Global Illumination at Pixar
DS: Visual effects primarily involve photo-realistic visual productions, much different from feature film animation production. How different is working on animated features at Pixar from working on live action – visual effects driven features at ILM?
CH: Off the bat, I don't find it that different to be frank with you. I think even in live action and visual effects, the goal is to tell a story. It's not necessary to replicate reality, so the lights that you see on the set are not real lights either. They're just put there so that the characters, the actors will look good. So it's kind of the same to me. You need to focus the attention of the audience somewhere. An artistically driven approach to lighting is very strong at Pixar because we are a story driven company. Part of what can enhance a story in a shot in particular is the lighting, as well as the cinematography, the placement of the camera.
What I would say is a little bit different on the other hand is that in visual effects we're used to trying to capture data from what's there on the set. As a starting point, for instance, we would capture a 360 degree high dynamic range image, sort of a dome, to give us an idea of what the lights were that the DP had placed on the set to start with. We would cheat from there most of the time as well. But it gave us a really good starting point. We took that approach as well on Monsters, on The Blue Umbrella and on films that are that coming up next. But, we give them a special twist at Pixar where we can paint on this environment in real time and see the effect in the character. So even though we went more in a photographic way, we were still using a kind of painting approach in those tools, if that makes sense, so the artist could express really complex behavior by painting on this dome, this texture map and seeing the result on the character. That's one approach we took.
We could also start from acquired images. A lot of the turntables, a lot of the look development, we did them under IBL, high dynamic range images that I shot upstream so we could canonize the materials across the different assets. That's a very visual effects approach to a show. More and more you've seen visual effects studios doing animation lately, like Weta with Tintin and ILM with Rango. I'm not saying they're merging, but the approaches are becoming roughly the same. I mean granted Pixar has a long history of storytelling which obviously is very different than any other studio. But in terms of techniques, in terms of approaches to lighting, we are sort of moving towards the same direction right now. I hope that all these techniques are freeing the artists in the end because as I said they give them something really good, really fast. And so at that stage, they can move to being artistic, take liberties with those lights and express their shots, make them even better as opposed to spending time debugging something.
DS: What’s next for lighting technology development? What are the holy grails?
CH: Well, I think one of the big one that's coming up is trying to move towards real time. Not necessarily real time delivery of the final shots but interactivity for the tools so artists can really place light and see the effects of the lights directly. They can do animation with the light rigs without being slowed down by the tools. Luckily, we have lots of [powerful] hardware. There are GPUs and special CPUs that are really beefy now, so we can do a lot more than we used to be able to do 5 or 10 years ago. That’s one big thing. In terms of image richness and simulation of materials or lights, we can always do better. There are more light transports that we need to build into the system. But we need to model better, we need to texture map better as well. It's not just the light, it's not just the material. Everything has to work together. There is a design that needs to be there, there's an amount of detail that needs to be there. All that stuff has to sustain the story.
In the past, I worked a lot on Pirates of The Caribbean. For instance, at ILM, Davy Jones was a famous character that we did. The reason why Davy Jones worked in particular was that it was a whole design. There was this octopus thing. There was the voice of Bill Nighy and his performance behind it. There was all the detail we put in the model. Of course we had really complex light transport on the character as well subsurface scattering. We were rendering the eyes and everything, but the success of it was because we had all those elements together. I don't think I can tell you we need to do this new technique for lighting that's going to make the image so much better. We can't do that in isolation of not potentially re-addressing how we model or how we texture map, how we do all those other things at the same time. That's what fascinating about this field. It's a really a strong collaboration between all the different fields. Artists, engineers, modeling, animation, look development, storytelling and lighting have to work together. I think harnessing the power of the computer is one of the easiest things we can do. It's a hard problem but we can do it for sure. The rest is probably longer term.
Dan Sarto is editor-in-chief and publisher of Animation World Network.