Virtual CG Characters in Live-Action Feature Movies

An example of the inclusive approach to virtual characters is Scooby. © Warner Bros. No other uses are permitted without the prior written consent of owner. Use of the material in violation of the foregoing may result in civil and/or criminal penalties.

This article contains excerpts from the third editon of The Art of 3D Computer Animation and Effects by Issac V. Kerlow.

During the last year we have seen groundbreaking developments in the creation of entirely computer-generated virtual characters for movies. These recent examples give us a hint of what is likely to come in this area during the next few years. Making all-CG virtual characters poses multiple technical and creative challenges, and we will examine some of them here. But first, lets define what virtual characters are and take a look at what has already been done.

What is a Virtual Character?

I like to define virtual characters in movies as characters that were not recorded by photographic means and that do not exist in reality, but that look and feel real. The performance of these characters is created mostly with 3D computer animation and compositing tools. There are two major schools of thought regarding what is a virtual character for movies. The more inclusive approach considers any believable CG character, cartoon or realistic, as a virtual character. An example of this would be a character such as Scooby from the 2002 live-action movie Scooby-Doo. This character was stylized as a cartoon and looked clearly different from the human live actors, but its performance made sense with the physical rules of the movies reality as it interacted with the human actors. The other point of view regarding what makes a virtual character limits the definition to those characters whose look and performance rival the look and performance of human actors in live-action movies. The Gollum character in The Lord of the Rings: The Two Towers (2002), for example, looked, moved and behaved in a way that was very similar identical at times to the human characters in the movie. This later point of view would not consider the stylized CG characters in animated movies as virtual characters, since they are clearly not real and respond to criteria and rules different from those of CG characters in live-action movies. One could argue that in spite of the fact that Shrek or Mike and Sulley (from Monsters, Inc.) are all incredibly convincing characters, audiences can tell that these characters are clearly animated and therefore not real. Which of these two points regarding virtual characters of view do you agree with?

replace_caption_kerlow02_TT-Gollum.jpg

The Prehistory of Virtual Characters

Virtual characters in movies as we know them today started in 1981 with the virtual character Cindy created by Information International Inc. for the science fiction film Looker. This was probably the earliest realistic model of the full human figure. The rendering was simple, and the animation was so limited that the model only rotated in a turntable style and had no animated joints. The 1985 movie Young Sherlock Holmes featured a jointed, texture-mapped and transparent character reminiscent of a medieval knight. This unusual character was constructed of flat stained glass panels and his animation was somewhat limited. During that same year the sexy female robot from The Brilliance TV commercial set a standard of realism and performance that future feature films aspired to match. This virtual character, created by Abel and Associates using rotoscoping and keyframe animation, had convincing realistic motion as well as a seductive, and believable, personality. Motion capture became a significant technique in the creation of virtual characters in the short animation test, Dont Touch Me, created in 1989 by the Kleiser-Walczak Construction Co.

The Early Examples

Some argue that only human-like CG characters qualify to be called virtual characters because of their potential depth of emotion. But I would argue that the performance of some CG-animated creatures is so convincing that they can also be included in our survey. The first example of a full-fledged, computer-generated, virtual character in a live-action film was Industrial Light & Magics gorgeous water creature in 1989s The Abyss. One of the most striking moments in this film takes place when the creature emulates the facial expressions of the human actor, who also touches the virtual creature with her hand.

Shrek and Mike and Sulley: Real or real-like? Shrek courtesy of DreamWorks Pictures. Monsters, Inc. © Disney/Pixar. All rights reserved.

Unlike todays virtual characters, the early counterparts in feature films created during the early 1990s had very limited screen time and limited performances. At most these early virtual characters had just a handful scenes of screen time and typically their performances were action-oriented, and silent to avoid lip-synching. Because of these limitations, experts argue that many of these creatures are more sophisticated visual effects than virtual characters, especially because most lack the emotional range that human characters can convey. But in spite of these limitations, some of these characters were nevertheless impressive at the time and some are still memorable. The main character in Terminator 2, the 91 landmark James Cameron film boasting computer animation by ILM, for example, was the first convincing simulation of natural human motion; it featured innovative three-dimensional morphing effects, and was rendered with global reflections, and even a few self-reflections in the scene when the virtual actor walks through the metal bars.

Steven Spielbergs The Lost World: Jurassic Park (1993), also done by ILM, is another example of early virtual characters using inverse kinematics skeletons with attached skins, local deformations for simulated muscles and hyperrealistic rendering. Jurassic Park was also the first example of a computer-generated human stunt double (the man in the portable toilet that gets eaten by the T-Rex). Digital doubles can be considered minor virtual characters since they look and act in a realistic way, but their performances are limited. Digital human doubles have become a staple technique that can be seen in most action movies, including Titanic (1997), The Matrix (1999), Pearl Harbor (2001), xXx (2002) and Harry Potter and the Chamber of Secrets (2002). CG-stunt doubles are usually animated with motion capture, as was the case, for example, with the digital extras on the deck of the Titanic ship.

Recent Virtual Characters

The end of the 1990s and the early years of the 21st century exploded with movie productions that sought to incorporate CG-animated virtual creatures. Most of these can be catalogued in three groups: the virtual attackers (usually monsters), the virtual heroes (usually human) and cartoon characters in a live- action world.

replace_caption_kerlow05_reignOfFire.jpg

Virtual attackers from this period usually include beasts or monsters that oftentimes attack as a group. Consider, for example, the morphing female character in Species (1995); the wild animals in Jumanji (1995) and their large scale of rendered fur; the masses of intergalactic bugs in Starship Troopers (1997) and Pitch Black (2000); the evil robot in Sphere (1998); Godzilla (1998); the running corpses in The Mummy (1999); The Lord of the Rings: The Fellowship of the Ring (2001), with its artificially intelligent crowds of virtual characters; the flying dragons in Reign of Fire (2002); and the underworld monsters in Lara Croft: The Cradle of Life (2003).

Virtual computer-generated heroes from these years are usually intercut with the human live-action versions of the character; this practice requires a high degree of realistic rendering. In Dragonheart (1996), the computer-generated dragon was, for the first time ever, the co-star of the movie. Remember the dramatic sequences in Spawn (1997) that combined realistic rendering with exaggerated keyframe animation; Mighty Joe Young (1998), with both the hyperrealistic virtual version and the rubber-suit version; the main character in Stuart Little (1999), with its innovative combination of cartoon and realistic action; the twisted scientist in Hollow Man (2000); and the short sequences where the human character was virtual to perform acrobatics in Blade 2, Daredevil, Terminator 3: Rise of the Machines (2003), Spider-Man, Star Wars: Episode II - Attack of the Clones (2002) and X2: X-Men United (2003) follow a newer trend where both heroes and villains are virtual.

The first two installments of The Lord of the Rings trilogy, The Fellowship of the Ring and The Two Towers, showcased stylized environments and creatures created with myriad techniques, ranging from in-camera effects to computer animation. Of special note are the eerie and emotionally convincing Gollum character, the crowd animation system and the superb digital color timing.

Cartoon and realism come together in Hulk. © 2003 Universal Studios. All rights reserved. Credit: ILM.

Cartoon characters in a live-action movie belong to a genre that goes back to Hollywood movies from the 1940s. Some of the most notable examples of this style include movies where the virtual characters are a unique mix between cartoony and realistic. Think, for example, of the amusing slapstick comedy of Mars Attacks! (1997); the wackiness of the Men in Black (1997) space aliens; the charm of the half-realistic half-cartoon characters in Stuart Little 2 (2002); the hyperrealism of the simulated wet cloth and wet fur, as well as the drama and comic book larger-than-life qualities of Hulk (2003); and the dark and gory humor of the ghost pirates in Pirates of the Caribbean: The Curse of the Black Pearl (2003).

The Future of Virtual Characters

As productions get more ambitious, virtual characters will increasingly require that their look and performance are convincing, and that both make sense within the reality of the live-action world. The success of future virtual characters depends on a few key creative and technical considerations. From the production point of view there are several issues and stages of the process that directly impact the characters look, including styling, modeling, texturing, rendering and compositing; rigging and animation that influence their overall performance. Rigging, for example, is an issue that clearly demands renewed attention, as a number of recent virtual characters (such as Gollum) required motion rigs that could be controlled by both keyframe and motion capture data. Currently, Warner Bros. Polar Express, a movie in production at Sony Imageworks, makes use of extensive motion capture as the main performance driver of virtual characters. On the animation front, the use of simulation software to take care of secondary action is likely to become increasingly necessary to deal with the complexity and enormity of the data involved in animating virtual characters.

Many Agent Smiths appear courtesy of CG in The Matrix Revolutions. © 2003 Warner Bros. Ent. U.S., Canada, Bahamas & Bermuda. ©2003 Village Roadshow Films (BVI) Ltd All other territories (All rights reserved used by perm

From the creative point of view, future virtual characters will need to deal with a wide range of challenges in order to be believable. Consistent and frequent interactions with live actors, full mobility and camera framing ranging from headshots to full body shots are some of the important factors on the performance front. Rendering techniques that approximate the subtlety of skin, eyes, hair and fur. The creative and production crews will gladly receive animation controls that streamline the animators job. Last but not least, future virtual characters will need to develop distinct personalities that can be as convincing as those of human characters.

Isaac V. Kerlow is director of digital production at The Walt Disney Co. in Los Angeles, California. He is also a longtime active member of SIGGRAPH and the Visual Effects Society.