Craig Zerouni looks at the state of CG and wonders if weve reached the end of significant technological breakthroughs in the medium.
Once upon a time, creating digital images using a computer was a difficult, obscure craft. Back then, there were a small number of companies devoted to doing CG. All of these companies were filled with Renaissance Men, programmers who wanted to be artists, sculptors who wanted to write software, refugees from pure science and fine art and cinematography, who saw that here was a medium that allowed indeed, forced them to use both halves of their big brains. The computers were also large, and in order to get anything out of them, people had to write software. Lots and lots of software.
Now, of course, this doesnt seem necessary. Virtually everything a budding CG artist or company could want is available off the shelf. There have been approximately 1 billion articles published detailing all the inner secrets of NURBS, ray tracing, radiosity, expanded dynamic range and whatever else is necessary. And as the math has receded, people with art school backgrounds have come flooding in to take up the mice and tablets. You can buy books explaining how to use any number of 3D applications, the applications themselves do amazing things at the push of a button (or 12), and it all runs on a laptop so cheap they give them away when you buy a ball point pen.
The current software is capable of maybe 80 or 90% of all effects shots without the need for a programmer. Products that were initially dismissed as interesting toys, are now used to do real shots on real movies. The dividing line between entry level and professional is very, very fuzzy.
Of course, there are still companies supporting teams of programmers, but vastly fewer of them (companies and programmers) and the pure R&D done by these teams is not extensive. And sometimes, the in-house guys come up with something, only to see the latest release of Brand X do the same thing better and more easily. Its getting harder and harder to differentiate a company purely with technological innovation.
And we can see this at SIGGRAPH. For most people, recent SIGGRAPHs have been underwhelming in terms of the new technology on display. For years now, its been about improving user interfaces or adopting better display hardware. But fundamental breakthroughs have been rare.
So it seems that digital effects is now, mostly, a known quantity, meaning that everyone knows how to do everything. We might reasonably ask if this is about it if maybe there are no more substantial technical breakthroughs to be had. Is the future of this business just about getting smaller, faster and cheaper? And where does this leave the large facilities? Up to now, theyve been able to differentiate themselves by having newer, more cutting-edge technology than the small shops. Is that advantage going to go away?
Well, its not over for them just yet. There are still things that are difficult to do, and there are still advantages to maintaining a large team that creates and maintains your own technology.
Lets start with some simple prognostication. Its fairly easy to predict more of the same: computers are still going to get faster and smaller and cheaper. We already see that techniques that were merely theoretically interesting, such as global illumination, have now become required, as hardware has caught up to the equations.
But what is happening is that more computer power is enabling the creation of more data such as various passes than can be coped with. When productions are generating 600 elements per frame, as now sometimes happens, there needs to be new tools developed to manage the result. And in general, there is a lot of ground to cover in collaboration tools databases that are effects aware and can help users navigate thousands of elements to find the three that they need, for example.
These are the things that big teams do: they create an infrastructure that makes it efficient (or even possible) to handle large numbers of shots, lots of layers, lots of workstations, etc. And now Discreet has given us the tragically named Toxik, which is a compositing solution with collaborative tools built in.
But these things are not CGI breakthroughs. At some level, these are management breakthroughs, or business model enhancements. And while we certainly need those too, nobody is going to write a SIGGRAPH paper describing how to organize a lighting pipeline for maximum throughput. So, what about real imaging technology?
Well, there are still areas waiting for real science to produce real breakthroughs. Natural phenomena are still hard to control, hard to get to look right, hard to make interact with the rest of the elements in the shot, and long-winded to compute. Water simulation, for example, is something that is still the subject of ongoing development all around the globe. Not only is water expensive to compute, it turns out that everyones notion of a water solution is something different. So getting spray and foam to work properly, getting water to interact with and motivate other geometry, getting water to dissolve things, getting it to look real while behaving controllably these are still hard problems.
Similarly, matching to wild footage is hard, and, in particular, extracting 3D data from 2D shots is a difficult problem that, if solved, would clear up a lot of logjammed pipelines. Not just extracting camera moves, but also extracting 3D locations and actual geometric elements the ability to do these reliably would be invaluable. Along the same lines, it would be slick to be able to present a sequence to a piece of software, and have the result be a reasonable guess as to the placement of lights in the shot, based on highlights and lighting falloff as the camera moved around.
Rigging characters is still an effort, especially characters that are non-standard; things with four arms and nine legs and two heads and a tail, for example, take a long time to get right. Better tools would be useful in this area.
Complexity itself remains difficult. Shots are now being created with upwards of 20 million particles. Some CG creatures now have 50 million individual hairs. These are big numbers, and coaxing them through even todays technology is hard work. We need to make that much more routine in the future.
Its also still a big deal to be able to close the loop on colors being able to see a color on a monitor, render it out and print it on film, project the film and then see the same color. This is a hard problem, and most shops muddle through on some combination of schoolboy science and luck. Right now, it feels like this problem will really be solved approximately one week before film is finally phased out.
Which is itself a major innovation. We are certain to get new, better displays, including something that looks like 3D but doesnt require special glasses or environments. And if studios want to keep people from just staying home and watching DVDs, they are going to have to find something special in presentation technology to get them back into the theaters. IMAX, or something like it, may be the sort of thing, in which case the technology of producing and delivering film effects will have to change too. Digital cinematography, always coming, never quite here, will have a profound impact on the technology of color, image storage and content delivery.
And then there is the big kahuna: artificial intelligence. In the early 1980s, Gary Demos argued that what we wanted to build was a system where we could say the cat jumps off the table and thats what would be rendered for us. Then we could spend our time lighting it well, or modifying the ways in which the cat jumped (Was he frightened? Did he pounce on an unsuspecting mouse?). You might think that, if such a thing existed, it would mean the end of animators. But thats never true. Most of effects is about iterating toward the best possible solution, and there is never enough time so if you can get the first 90% out of the way in 10 minutes, that leaves the rest of the week to get to 99% of what the director wants.
I can imagine that, apart from the obvious problems with recognizing natural language sentences and turning them into something meaningful to an animation system, there are going to be subsidiary problems of control. For example, once you are able to speak into your mouse, and tell it to animate a cat, you are going to expect to be able to modify it the way they did in Blade Runner: Left a bit. Stop. A little more exasperated, please. No, too much. Perhaps defining exasperated will be something you create as a plug-in.
We already have libraries of animation cycles. If we can marry those to emotional states, or motivation, then we can start to direct CG humans as well. I dont believe that we will ever have photoreal CG characters as the stars of films, except for novelty value, but clearly we have already passed the threshold for replacing actors with CG stand-ins for lots of different kinds of shots. In many ways, this will soon pass from a technical problem to a business one. Perhaps the cost of talent will motivate studios to replace $20 million stars with $2 million CG stars. Perhaps.
So there are still some areas where enterprising eggheads can make a contribution, and Im absolutely sure Ive left off something that someone is working on right now, that will be indispensable five minutes after it is released on an unsuspecting world. Which is good, because we still dont have all the tools we need. In the future, studios will want what they have always wanted: something visually exciting enough to motivate people to leave the house and go to the theater, and theyll want it to cost less and be done faster. To achieve this, they are more and more willing to use small shops. Small shops are good at the quick, cost-effective part. However, they are less likely to develop new visual technologies.
The new technologies are less likely to be earth-shattering leaps forward, but they will be necessary just the same. Its always a mugs game to predict the end of technological improvement, and we shouldnt start now.
Craig Zerouni is currently a production consultant with Side Effects Software, developer of the award-winning Houdini family of 3D software. Zerouni collaborates directly with production studios and technical directors to help build superior CG pipelines and create cutting-edge digital animation for feature films. The 20-year veteran has served as technical director with Rhythm & Hues ( Daredevil). Through his work with CFX Associates and Silicon Grail, Zerouni consulted on First Knight, Godzilla, The Prince of Egypt and Hercules, among others. He has also supervised the vfx for hundreds of TV commercials.