Scientist/filmmaker Dr. Paul Debevec has been creating photoreal imagery since he was an undergraduate at the University of Michigan in 1991. In what is now known as image-based rendering, Debevec created a realistic three-dimensional computer model of his beat-up old Chevette from six photographs. Continuing to develop techniques for improving 3D modeling and lighting, his Ph.D. thesis at UC Berkeley — a geometric reconstruction technique for modeling architecture from photographs — resulted in practical applications: photoreal three-dimensional models of the UC Berkeley Campus, the Rouen Cathedral and St. Peter's Basilica.
The "scientist" is currently a research assistant professor at the USC Viterbi School of Engineering department of computer science and exec producer, Graphics Research at the Institute of Creative Technologies. ICT is charged with developing technology that includes artificial intelligence — digital characters that react to situations like real humans would react and immersive learning environments — advanced virtual reality and simulation technology — of interest to the U.S. military for training troops worldwide. Many of Debevec's projects have produced research publications, including several SIGGRAPH papers.
The "filmmaker" has won awards worldwide for computer-animated shorts using his cutting-edge CG modeling and rendering techniques. Technology developed by Debevec was used to achieve groundbreaking feature film effects such as the "Bullet-Time" shots in the Matrix films. When the filmmakers needed to cut into bullet-time, a photoreal CG version of the set allowed them complete control over virtual camera movement within the set and the ability to seamlessly line up with the actors that were filmed with the time-slice array. His experiments with virtual backgrounds and image-based lighting are "things I worked on early enough that's there been time for people to do really cool stuff with it, particularly in the visual effects industry," explains Debevec. "That's one of the neatest things about doing research that's relevant to the visual effects industry because they're the perfect early adopters of leading-edge technology." For Debevec, his collaboration with vfx artists in the entertainment industry is a natural and mutually beneficial extension of his academic and scientific work. It allows him to bring together top technical and creative talent working towards a common goal. The artists stretch the technology to its limit and beyond, indicating where the next research frontier is.
Debevec's 3D modeling techniques are based on using CG primitives — boxes, wedges and other geometric shapes — with photographic images in a way that the computer can figure out the 3D re-construction of the photo. His photogrametric technique — Facade — makes computer graphics primitives snap to the edges in photographs, so the computer can use perspective geometry to quickly create a full three dimensional structure of the scene. Once the computer recreates the geometry, the photographs are projected onto the model as texture maps. Debevec wants to create a whole environment, but unlike a virtual setting like the storybook appearance of the Polar Express, he has a different goal. "Our work is focused to make it look real, as real as photographs." But in 3D modeling, getting the right geometric representation of an object or scene won't make it photorealistic. Lighting is an inescapable factor in making CG images believable.
Debevec learned lighting while taking photographs of architecture. He noticed the different effects in varying amounts of sun and shade. Photos and computer images have a limited range of colors and pixel values while most film scenes have a wide range of varying light conditions. Realistic lighting not only relies on direct lighting sources such as the sun, but includes the reflective properties of objects. Debevec knew he needed better tools. "These 8-bit pixel values really weren't good for representing anything brighter than what a monitor could display or dimmer than where things start to look muddy. It became painfully obvious that the range of light in the real world is far greater than this." Debevec worked to capture the full range of light by taking a series of photographs with variations of exposure that when put together in a final image, balanced the levels of brightness and mimicked lighting in the real world. This work with High Dynamic Range Imagery (HDRI) contributes to making CG images look absolutely real. The technique is used to record the illumination on a set and then light a digital character, that when added to the scene, would actually be lit with the light that was really on set.
This August at the SIGGRAPH 2004 Conference, Debevec's cutting-edge computer graphics technology will be showcased in his latest short film, The Parthenon. The film premieres at the prestigious Electronic Theater Computer Animation Festival at the Los Angeles Convention Center. Using the best archaeological sources available, The Parthenon takes the viewer on a two-and-a-half minute virtual journey through both the digitally reconstructed temple and a photorealistic recreation of the Duveen Gallery of the British Museum, home to a majority of the Parthenon sculptures.
The crowning monument of the Athenian Acropolis in Greece, the Parthenon was dedicated to the goddess Athena in 438 BC and remains one of the world's greatest architectural achievements. Its sculptural decorations included marble statues in its triangular pediments, high relief metopes above its outer columns and a low-relief frieze that circled the inner colonnade of the temple. Over the years, it has been battered by time and wars, damaging or destroying some of these details. Most were removed in the early 1800s when the British Lord Elgin had them shipped to England. They remain on display at the British Museum despite requests from Greece for their return.
In the film, Debevec and his ICT research group use technology to virtually return the Parthenon's sculptures to their former places. To achieve this simulation, ICT developed two custom scanning processes — one for the sculptures, one for the architectural ruins. The team traveled to Switzerland's Basel Skulpturhalle, a museum that houses high-quality casts of all the Parthenon sculptures and fragments scattered throughout Europe, including the ones in London, Athens and Paris. More than 2,000 scans of the collection were captured using a newly developed ultrafast 3D scanning system that allowed highly detailed geometry to be recorded with a standard desktop video projector and digital video camera. These were then assembled into millimeter-accurate models of the friezes, metopes and pediment sculptures.
A different technique was used to capture the massive — 73 meters in length — Parthenon. Working with the ongoing Acropolis restoration effort, the team used a time-of-flight 3D laser scanner to record the geometry of the temple. The device uses an infrared laser and spinning mirror to scan an invisible spot of light across the surface of the structure, determining the distance to each point by measuring the time for the laser to return. The panoramic laser scans in and around the Parthenon — comprising more than six billion 3D point measurements of the site — were then used to create a 90-million polygon model of the Parthenon and its surrounding environs.
Digital photography of a structure only shows the shading, highlights and shadows under the illumination present when the photograph was taken. The coloration of the Parthenon's weathered surfaces was achieved by a surface reflectance estimation technique. A "light probe" device measured incident illumination from the sun, the sky and clouds at the same time that the digital photograph was taken. When the photographs were unlit in the laboratory, the lighting effects were corrected, revealing the true colors of the surfaces. This technique is accurate to within 5% of the actual surface colors — the most accurate method to date.
The last element necessary to complete the photorealistic reconstruction of the Parthenon was realistic lighting. Lacking the time to wait in Greece for the combination of clouds and sun that he needed for just the right illumination, Debevec used lighting captured from the roof of L.A.-based ICT to form the virtual lighting in their renderings of the Acropolis.
The result of all this advanced technology is a computer-generated film of breathtaking realism and artistic achievement. In the film the two sites — the temple and the museum — merge. Clouds roll across the sky with tumultuous magic as the marble sculptures reappear in their original niches and ledges in the temple. The light changes, reacting as the sky changes from cloudy-bright to direct sun. In the final sequence of the film, the temple's original bright painted colors fade-in, restoring the Parthenon and its sculptures to their ancient glory — a sight not seen for millenniums.
Mary Ann Skweres is a filmmaker and freelance writer. She has worked extensively in feature film and documentary post-production with credits as a picture editor and visual effects assistant. She is a member of the Motion Picture Editors Guild.
Links:
[1] http://www.awn.com/imagepicker/image/7650
[2] http://www.awn.com/imagepicker/image/7651
[3] http://www.awn.com/imagepicker/image/7652
[4] http://www.awn.com/imagepicker/image/7653
[5] http://www.awn.com/imagepicker/image/7654
