Harvard SEAS Advances Digital Imaging
“This study, aiming to understand the appearance space of phase functions, is the tip of the iceberg for building computer vision systems that can recognize materials,” says Zickler. The next step in this research will involve finding ways to accurately measure a material’s phase functions instead of making them up computationally, and Zickler's team is already making progress on this, with a new system that will be presented at SIGGRAPH Asia in December.
Zickler’s coauthors were Ioannis Gkioulekas, a graduate student at Harvard SEAS; Bei Xiao and Edward H. Adelson of MIT; and Shuang Zhao and Kavita Bala of Cornell University.
A second study involving Zickler investigates a new type of screen hardware that displays different images when lit or viewed from different directions.
By creating tiny grooves of varying depths across the screen’s surface, Zickler’s team created optical interference effects that cause the thin surface to look different when illuminated or viewed from different angles.
The paper essentially asks, “If I know what appearances I want the screen to have, how do I optimize the geometric structure to get that?” Zickler explains.
The solution takes advantage of mathematical functions (called bidirectional reflectance distribution functions) that represent how light coming from a particular direction will reflect off a surface.
Past attempts to control surface reflection for graphics applications have only been accomplished for surfaces displaying huge images that, for example, have pixels the size of a square inch, because their analyses did not account for interference effects. Zickler’s work, however, demonstrates that interference effects can be exploited to control reflection from a screen at micron scales using well-known photolithographic techniques.
In the future, this kind of optimization could enable multi-view, lighting-sensitive displays, where a viewer rotating around a flat surface could perceive a three-dimensional object while looking at the surface from different angles, and where the virtual object would correctly respond to external lighting.
"Looking at such a display would be exactly like looking through a window," Zickler says.
He was joined on this paper by Ying Xiong, a graduate student at Harvard SEAS; Anat Levin and Daniel Glazner at the Weizmann Institute of Science; and Frédo Durand, William Freeman, and Wojciech Matusik at MIT.
A third paper, led by Hanspeter Pfister, An Wang Professor of Computer Science, tackled a problem in digital film editing.
Color grading—editing a video to impose a particular color palette—has historically been a painstaking, manual process requiring many hours’ work by skilled artists. Amateur filmmakers therefore cannot achieve the characteristically rich color palettes of professional films.
“The starting idea was to appeal to broad audience, like the millions of people on YouTube,” says lead author Nicolas Bonneel, a postdoctoral researcher in Pfister’s group at SEAS.
Pfister’s team hopes to make frame-by-frame editing unnecessary by creating software that lets users simply select, hypothetically, the Amélie look or the Transformers look. The computer would then apply that color palette to the user’s video via a few representative frames. The user only has to indicate where the foreground and background are in each frame, and the software does the rest, interpolating the color transformations throughout the video.