Search form

Shine On: Global Illumination and 3D Environments

Filmmakers are always raising the bar on what vfx artists need to create, especially in the area of photorealism. Janet Hetherington chats with professionals about the newest mandatory effect: global illumination.

The gleam of the glass on the towers in Batman Begins reveals how global illumination adds to the photorealism. All Batman Begins images courtesy of Double Negative. © 2005 Warner Bros. Ent. All rights reserved.

The gleam of the glass on the towers in Batman Begins reveals how global illumination adds to the photorealism. All Batman Begins images courtesy of Double Negative. © 2005 Warner Bros. Ent. All rights reserved.

From Wayne Tower in Batman Begins, to a magically controlled spider in Harry Potter and the Goblet of Fire, to an array of imaginative environments in King Kong, Revenge of the Sith and the upcoming Monster House, vfx pros are using global illumination to trip the light fantastic and make the unreal look convincingly real.

Global illumination algorithms used in 3D computer graphics take into account both direct illumination (the light which has taken a path directly from a light source), and the indirect illumination (the light that has undergone reflection from other surfaces in the world), when considering light falling on a surface.

Global illumination has made a huge difference to the approach to creating 3D lighting effects, says Paul Franklin, visual effects supervisor and head of 3D at London-based Double Negative (Batman Begins, Harry Potter and the Goblet of Fire and Doom). The technique has made a major contribution to the ever-increasing realism of 3D environments seen in films today.

Franklin adds that before the advent of practicable global illumination, most 3D environmental lighting was produced through a series of elaborate cheats, which would work for a few hero shot angles, but which would generally break down if the camera moved around too much. Global illuminations use of optically and physically correct simulation helps to produce a continuous lighting environment that behaves in a consistent and naturalistic way, he suggests.

We started using GI techniquesmainly ambient occlusion and ambient reflectionat the beginning of Spider-Man 2, and the difference that made in our environments compared to Spider-Man is clearly visible. Ever since then we have kept refining those techniques, comments Peter Nofz, digital vfx supervisor, Sony Pictures Imageworks.

The challenge is always balancing realism with cinematic drama, adds Jay Redd, senior vfx supervisor, Sony Pictures Imageworks. We are now to the point in the industry where we can create any environment we want. And while Redd admits that visualization of the 100% photorealistic dramatic performing human has not yet been achieved, he notes, With enough work, we can essentially create any character we want. The advantage in using GI is that you get a lot of stuff for free. Instead of having to go through a scene and placing perhaps dozens of lights to get the desired balance of light fill, bounce, rim, highlight, kick, etc. we can now cut that in half. The truth is, you could put one light in a room, turn on GI and get an interesting image, complete with bounce and color bleed, but is it dramatic or cinematic? GI may allow us to get there faster, but we still have to get there in a creative and interesting way. We have to make each scene and character its very best for the story.

In King Kong, for instance, global illumination helped light the New York sequences and enhance the period look. "We did a whole lighting progression, admits senior visual effects supervisor Joe Letteri. We went from pre-dawn to about 10 minutes after dawn that all happened in sort of realtime. We needed all the bounce lighting."

Escaping the GI Ghetto

A common approach for vfx artists is to compute the global illumination of a scene and store that information with the geometry. As Redd suggests, that stored data can then be used to generate images from different viewpoints for generating walk-throughs of a scene without having to go through expensive lighting calculations. However, while images rendered using global illumination algorithms are more photorealistic, they have traditionally been much slower and more computationally expensive.

This is changing. The basic ideas behind global illumination techniques have been around for a long time, but in the past, they could not be used in film production because they were too slow to produce images, affirms Nofz. The real breakthrough happened with the speed increase of current CPUs, making it finally possible to use GI effectively. The other breakthrough happened in software refined algorithms started to surface that would handle specific problems in a more efficient manner.

Double Negative has been utilizing global illumination on many of its recent projects, including the Mars exteriors in Doom. Courtesy of Double Negative/Universal Pictures.

Double Negative has been utilizing global illumination on many of its recent projects, including the Mars exteriors in Doom. Courtesy of Double Negative/Universal Pictures.

For a number of years global illumination was definitely in the cool buzzword ghetto one of those terms that would get bandied about without many people really understanding what it meant and just how difficult it was to use it, asserts Franklin. However, now that software has improved and hardware speeds have dramatically increased, it has become very much a standard part of the CG lighting arsenal.

Beginning with Star Wars: Episode III Revenge of the Sith, this has been the case at Industrial Light & Magic. CG traditionally handles surface illumination, says vfx supervisor John Knoll. But interior environments are completely dominated by indirect nodes. Those kinds of effects can be simulated through the technique of radiosity [a subset of full GI]. Weve started incorporated that into our methodology here so we can achieve more realistic environments.

Adds ILM colleague Jonathan Harb, digital matte supervisor: Whats great about global illumination is that we dont use it in a typical way in the matte department. What we want to know from global illumination is what is the quality of the light in a building? What is the quality of the highlight like? What is the quality of that transition across its face like? But we dont want to render it for every frame because its very processing intensive. Well render one frame that is representative of what we want.

Franklin says filmmakers are always raising the bar on what the vfx artists have to create, particularly in the area of photorealism. If a serious vfx artist is looking to develop a killer digital set extension or 3D environment, then some kind of global illumination solution is pretty much mandatory.

GI may be fast becoming a must-have, but its still an emerging technology. According to Redd, Future projects demand more efficiency as schedules and budgets shrink. Hardware gets faster and cheaper, and new algorithms are developed, new efficiencies are found, pushing the industry to the next level. If you were to search the Internet for GI software, you could find applications for virtually every hardware platform on the market PC, Mac, Linux, etc. I think the key is how the GI solutions are implemented. These solutions are many: Monte Carlo, photon mapping, irradiance caching, ray tracing, etc. Take SIGGRAPH, for example; each year, more and more attention is devoted to efficient global illumination and radiosity techniques.

The hardware is the same type of processors and workstations we already use for all our rendering needs, adds Francisco X. DeJesus, CG supervisor at Sony Pictures Imageworks. The only thing that is different is that since our internal global illumination renderer was written with multithreading support, we do prefer to run renders on machines with multiple processors or the newer processors with multiple cores as we get a very big reduction in render times that way. As multiple cores become the norm, there will be more value in applications that can take advantage of multithreading.

Franklin says a multi-processor render farm is often required to handle the large computational demands of global illumination. Software-wise, many commercially available rendering packages now include some aspect of global illumination type lighting: mental ray, for example, supports photon maps amongst other things. At Double Negative, we have a strong R&D team that writes a lot of in-house lighting tools, which are usually integrated into our RenderMan pipeline. We also have a couple of our own proprietary renderers, such as the DNB volumer renderer, which support various global illumination techniques.

Franklin also says that Double Negative has developed a robust set of asset management and publishing tools that cover all aspects of the 3D pipeline right down to lighting rigs and HDRI environments. The amount of data created by techniques such as global illumination requires strong organization and good design of work flowif you don't do this then you can end up in a mess very quickly!

Too expensive to use on Spider-Man, global illumination has advanced so much in a few years that Sony Pictures Imageworks found it a viable solution on Spider-Man 2. © 2002 Columbia Pictures Industries, Inc. All rights reserved.

Too expensive to use on Spider-Man, global illumination has advanced so much in a few years that Sony Pictures Imageworks found it a viable solution on Spider-Man 2. © 2002 Columbia Pictures Industries, Inc. All rights reserved.

GI Cost Factor

Cost, however, remains a factor. Because more machine power is needed and the algorithms involved are more complex, global illumination is more expensive than traditionally rendered CG, says Nofz. There are substantially longer render times, a need for more RAM and disk space, and all these factors contribute to raising the cost.

Most of the costs are incurred from R&D, notes Redd. Solid algorithms need to be implemented. Special looks need to be created. Efficient workflows and pipeline interfaces need to be created and tested. All of these things take people and time. Costs are coming down, but I would attribute a lot of that to faster processor speeds. It's now way more feasible to use GI solutions than it was even three years ago.

The amount of processing time required is the biggest challenge to GI, adds DeJesus. The second biggest consideration is how it can be art-directed. We have tried to use radiosity for large environments going as far back as the first Spider-Man, but would always find the results too costly, or not flexible enough when we wanted to cheat or direct what came out of the box for dramatic effect. So we would end up approximating the look with lots and lots of direct lights, and lots of shadow maps. That in itself can quickly become a nightmare to manage and keep track of.

"On more recent films, we have adopted the use of ambient occlusion as another step towards photorealism. Depending on the renderer, though, ambient occlusion can also be quite costly, and we developed a workflow that allows for this to be rendered as a separate pass once, and avoid re-rendering it for subsequent lighting iterations.

The key factor to consider with regard to cost, according to Franklin, is how long will it take for artists to set up? A manually driven approach (i.e. artists placing lights) can produce interesting results very quickly, but often ends up very slow and unmanageable as the complexity of the scene increases. Global illumination takes a lot longer to set up, at least for a complex environment, but the high level nature of the controls used can allow you to make limited changes to complex scenes in a timely fashion. The great thing is that as software and hardware advance, the costs of the technique are dropping all the time.

GI in the House

The cost of manpower and computing power aside, what GI has to offer has made it an effect of choice. Columbia Pictures Monster House to be released next summer in both regular and 3-D formatsutilizes the same Imagemotion performance capture innovation by Sony Pictures Imageworks introduced on The Polar Express.

Monster House is the first project we are doing where global illumination is an integral part of the look, offers DeJesus. It is a full-CG show, but rather than looking at other CG movies as a point of reference, we started from day one looking at stop-motion films for the look we wanted to aim for. One of the many visual cues we picked up on when watching stop-motion films is that because those are typically miniature sets, some lighting phenomena such as light and color bounce is more prominent. This led us to looking for a GI solution.

For full-CG films, it may remain a stylistic choice since CG isnt and shouldnt be restricted to a singular look. But effects work on live-action films usually has a more established photoreal look that must be matched, and GI can help get closer to that. It is particularly well suited to live-action films where shots are composed of synthetic environments and characters, as opposed to just putting a single character in a photographed plate. That is because for GI to work well, its best to have everything in the environment geometry, textures, materials, etc. in the 3D scene.

The software we used [on Monster House], besides our internal global illumination renderer, was a mix of third-party and proprietary packages. We were able to insert this renderer into our existing facility pipeline without having to rework a lot of it. It took more work to build tools that worked with Maya to facilitate the workflow we developed as we solidified the look we wanted and the most efficient methodology for achieving it. Thankfully, we have a lot of experience in pushing large volumes of work through CG lighting and rendering and were able to leverage that knowledge while we were figuring out how to do an entire movie with GI.

Redd says they also wanted to increase their efficiency. Up to now, achieving a true bounce light look was cost and time prohibitiveOn this film, we wanted to create something special, something tangible something that would immerse the viewer into another world stylized, yet real.

For the all-CG world of Monster House, global illumination plays a significant role in bringing it vibrantly to life. © Sony Pictures.

For the all-CG world of Monster House, global illumination plays a significant role in bringing it vibrantly to life. © Sony Pictures.

We are creating a photoreal world, much as a doll-house, miniature set, claymation or a stop-motion film would be real its all about indirect illumination reflected light. Global illumination techniques allow my team and I to work like a director of photography would on a set. The physics of light actually mean something now. My team can act like key grips and gaffers. We get to utilize real properties of light; we get bounce, energy loss, color bleed, gels, gobos, etc. Much like a live-action world, the color of the wallpaper or carpet in a room really has an affect on a character. Every choice has to be conscious and calculated. Its very simple to over light or over light with GI. Because of the direct and indirect illumination, global illumination gives us a wonderful depth to even the simplest of geometries.

According to Redd, global illumination was key to the look of the characters as well. On Monster House, we are creating a unique hybrid of performance that is part keyframe, part performance capture and part stylized character design this is not so different from traditional puppetry. Add to this mix creating tangibility through the use of global illumination-based lighting, and suddenly we have something truly unique and unseen.

Our philosophy on Monster House is to make everything feel as if it has been hand-made made from something you might find at the local craft store. I sometimes like to think of our characters as the most articulate large-scale stop-motion puppets ever created. Global illumination is an absolutely essential part of our recipe for the look of the film.

More shots of Batman Begins stellar use of global illumination.

More shots of Batman Begins stellar use of global illumination.

More GI in Action

A key goal of filmmaking is to make the imaginary come to life and make it seem real. In Batman Begins, the architecture of the mythical (and mythic) city of Gotham City had to be completely convincing. For the daylight scenes in Batman Begins, we used a variety of lighting techniques including global illumination, which was very important in creating the two shots showing the train approaching Wayne Station at the centre of Gotham City, says Franklin. The first of these shots combined 3D-rendered elements with live-action plates of Chicago architecture; the second shot was entirely 3D-generated. Proprietary global illumination tools developed by the Double Negative R&D department ensured that the lighting of the 3D architectural scene blended seamlessly with the reality of the preceding shot.

There is a specific shot in Batman Begins of Wayne Tower that I am particularly fond of. It precedes a sequence set inside the Wayne Industries Board Room and used a complex global illumination tool set to create a fully CGI Wayne Tower that integrated perfectly into the live-action plate of Chicago. The complexity and subtlety of the lighting effects on the front facade of the tower are a great example of strong creative lighting being applied through a powerful, state-of-the-art technique.

Global illumination can be equally powerful in creating totally imaginary environments. For the upcoming film Flyboys, produced by Dean Devlin, Double Negative artists created sweeping aerial panoramas featuring "architecturally arranged" cloud formations, Franklin continues. The lighting of these clouds required a proprietary global illumination approach to create the necessary effect and to properly light the complex aerial battles between them.

For the film Doom, Double Negative created the exteriors of the Martian Station. This was a totally created environment that only ever existed as 3D CGI. Global illumination was vital in creating a sense of believable reality in an otherwise completely fictional setting, Franklin adds.

Double Negative used global illumination for the creepy spider scene in Harry Potter and the Goblet of Fire. © 2005 Warner Bros. Ent. Inc.  Harry Potter publishing rights © J.K.R.

Double Negative used global illumination for the creepy spider scene in Harry Potter and the Goblet of Fire. © 2005 Warner Bros. Ent. Inc. Harry Potter publishing rights © J.K.R.

GI also played a key part in a very different sequence that Double Negative created for Harry Potter and the Goblet of Fire one that featured the spider crawling all over the students in a Hogwarts classroom. The primary 3D component of the shot is the spider itself, global illumination techniques were used to create the lighting contribution of the class room environment to the spider allowing it to be precisely integrated with the live action, Franklin explains.

GI Reflections

As global illumination becomes entrenched in the world of movie magic, vfx professionals are building on the techniques strengths and weaknesses.

The key advantage of using a global illumination approach is that you are building a simulation of the way that light energy is exchanged between all the objects in your scene each object makes some kind of contribution depending upon its size, position, color etc., suggests Franklin. As a result you get some very complex lighting interaction that would otherwise be very time consuming to create by manually placing lights. Global illumination can also take into account very subtle effects like atmospheric haze and smoke, procedurally creating very delicate lighting effects that would be difficult to achieve in any other way.

The downside of global illumination is that it can be difficult to manage in a creative, interactive fashion. As anyone who has worked in vfx knows, there always comes a point in any job where the client says, That looks great, but can you make the sea green/sky yellow/earth red but dont change anything else! And, of course, you cant do it that easily because all parts of the scene are linked to each other through the global illumination solution. Theres no easy answer to this kind of problem, but careful planning at the outset when designing your work flow can help.

We continually strive for better and more convincing imagery and towards this end, GI is instrumental, concludes Nofz. This is not just a trend. We will see more complex GI algorithms used as we move forward.

Janet Hetherington is a freelance writer and cartoonist. She shares a studio in Ottawa, Canada, with artist Ronn Sutton and a ginger cat, Heidi.