Force of Hobbit: Mythmaking at Weta

Mythmaking, moviemaking. These days, it's all the same. By now, you may have heard of a trilogy of movies called The Lord of the Rings. But perhaps, by some omission or oversight of conversation, you may not have heard of the studio behind the movie the Ring Leader, as it were, for the brilliant digital and practical effects that have helped to adapt J.R.R. Tolkien's literary masterpiece into a visual one.

We speak, of course, of none other than Weta, Ltd.

Situated unassumingly in a quiet, nondescript suburb of New Zealand's capital, right next door to an old weathered ice cream factory, Weta is a state-of-the-art facility producing some of the world's most fantastic visual effects. Even as we speak, an entire world populated by elves, dwarfs, hobbits, trolls, orcs, ents, wraiths and balrogs is patiently coming to life. (And you thought you had a busy workload.)

Peter Jackson, director/writer/producer of The Lord of the Rings. All images © 2001 New Line Cinema Productions.

For the first film of the trilogy, The Fellowship of the Ring, something on the order of 570 special effects shots were produced with some of the more complex shots having over 300 input layers of various types, and thousands of operations in the final compositing script. Now, with hushed devotion and tight security, work is already well under way for the second of the films, The Two Towers, due in theaters this coming December 2002. And the final film, The Return of the King (2003), will have effects work equivalent to the first two films combined.

For The Fellowship of the Ring, the number of artists and technicians at the Kiwi studio doubled in the year leading up to the film's wrap date, from 90 to nearly 170 artists at its peak. It is expected the staff will need to grow to 250 persons to complete the work on The Two Towers, slated for delivery in October of this year. While the practical filming for the trilogy is done, other than pickups, the artists will be soldiering on for the next 18 months, or so, to finish the two remaining films.

Director Peter Jackson and the Weta team have had to achieve, visually, in five years what Tolkien labored for 12 years and 1200 pages to create in words.

Fellowship: A Brief History of Weta

Fourteen and a half years ago, Richard Taylor and his partner Tania Rodger decided they wanted to start an effects business in Wellington, New Zealand. There was no such industry within the country at the time. Therefore, starting a workshop that specialized in models, puppets and physical effects was quite an unusual undertaking perhaps even more so given the rural upbringings of Taylor and Rodger. Taylor explains, "We don't come from a long history of loving effects films, and suchlike. We didn't have access to the movie culture that others may have grown up with." But Taylor and Rodger saw it as a great way to utilize the skills they had, and to devote themselves to their love of making things. The operation began in the back room of their flat as RT Effects, Ltd.

Richard Taylor, head of Weta Workshop, supervising special makeup, armor, weapons, creatures and miniatures.

Taylor and Rodger worked for two years, before meeting Peter Jackson. They struck up an instant friendship. Jackson employed them in their first collaboration on Meet the Feebles (which, for those who have yet to see it, is often described as "the Muppets on acid"). Following the strength of that project, Taylor and Rodger went on to supervise the creature, gore and model effects, as well as to do a small amount of stop-motion animation, for Braindead (a.k.a. Dead Alive). Jackson, Taylor and Rodger continued in their work together on Heavenly Creatures, based on a famous murder case in New Zealand from the 1950s. It was during the making of this latter film that they leased one computer to do the small amount of digital morphing and animation required of the story. As the production drew to a close, Taylor and Rodger realized that if they let this one computer return to its leaseholder in the United States, it would be a big loss for the visual effects possibilities within their country. So, in 1994, pooling their resources, Peter Jackson, Richard Taylor, Tania Rodger and Jamie Selkirk formed a company called Weta, Ltd., which allowed them to continue leasing the computer. Thus were Weta's humble beginnings.

"Weta" is derived from the name of a native New Zealand insect, from the cricket family, which has been around since before the dinosaurs. The weta is the size of the palm of one's hand, able to survive extreme conditions, even being frozen in ice, and it has the distinction of being the heaviest insect in the world. Although the name means nothing outside of New Zealand, Taylor acknowledges, it is a colloquial favorite. Taylor says, "It's the ugliest little monster on the planet, while still being the most beautiful insect. It was a great little creature to aspire to."

Within a year, the fledgling Weta team had set up both the physical and digital arms of their company. During the last six years, they have worked on numerous New Zealand films and foreign productions shot locally, including: Hercules: The Legendary Journeys and Xena: Warrior Princess; some shots for Robert Zemeckis' Contact; a King Kong remake with Peter Jackson, which has since been shelved; and, also with Jackson, The Frighteners and an hour-long mockumentary called Forgotten Silver.

Amongst all of this, Jackson came to Weta and said, "Hey, everyone, we're making Lord of the Rings," whereupon, according to Taylor, they promptly fell over. Picking themselves back up again, and brushing the dust from their sleeves, they delved into the making of the trilogy.

A multi-level composite shot, combining miniatures, matte painting, sky replacement and CG effects, while the foreground comes from a practical element filmed on a New Zealand landscape.

At this stage, Weta, Ltd. formed two companies, because each had grown in their own right. Weta Digital handles the visual effects work for the films, and Weta Workshop oversees the construction of physical sets, props and other practical effects. Richard Taylor is the supervisor looking after the Workshop, and Tania Rodger serves as the Workshop's manager. In all, Weta comprises 68,000 square feet, with five departments gathered under one roof, which is, in itself, quite unique within the world of special effects.

As Taylor explains, "We primarily chose to look after so many departments, so we could create a very cohesive vision for the picture. The written word of the author is coming from a singular vision, and we realized that if we dissipated the work around the world, the vision would also dissipate. The great relationship and benefit of Weta Workshop and Weta Digital in being under the same roof, within the same complex, is that it creates a very easy flow of information and vision to the overall project."

Weta Workshop manages special makeup effects and prosthetics, armor, weapons, creatures and miniatures.

On The Lord of the Rings, the Workshop accumulated and built over 48,000 separate pieces for the film -- including 2,000 weapons, 68 miniatures, 10,000 facial appliances, 10,000 arrows, 1,800 body and prosthetic suits, 1,000 suits of armor, as well as numerable other interactive creature effects, scannable maquettes, and the like.

A motion control camera careens around and up the 1/166 scale, hyper-detailed miniature of the Barad-dûr tower. In the film, Sauron's minions construct the looming tower by the light of a thousand torches.

The Weta team began work creating miniatures two and a half years before the start of production. "Peter utilized miniatures as huge animatic shooting models, and through them designed the shots and action that would be required on the full-sized sets," Taylor explains. "We coined the phrase 'biggatures' in the Workshop, as some of our miniature constructions rivaled in size some of New Zealand's largest live-action set constructions to date."

The "small part," as Taylor says, that the Weta Workshop played in the animation for the films was their development of the creatures. Beginning with the absolute philosophy that Tolkien's fantastical world of Middle-earth was based completely in a reality both possible and believable a kind of modern English folklore drawing from a huge reference of historical documentation Weta pursued, to every length, this same sense of realism. They built their own blacksmithing shop, literally beating the armor suits out of plates of steel, and hand grinding the weapons. Every prosthetic foot and ear was individually crafted, shaped and sized to fit the needs of the characters. The physiology of the creatures had been exactingly designed. Taylor notes, "We imagined and delivered to [the Weta Digital team] creatures that could have existed in our own world, with the same physical structure, dynamics and mechanics of muscle that a creature would have in our own time."

These Troll images show how detail obtained from the original clay maquette with the 3D laser scanner is applied to the finished rendered creature as an extracted displacement map.

To model the creatures, maquettes were sculpted in the Workshop at a very large scale, four or five feet tall, in the highest detail. Then, using a hand-held laser scanner originally developed for scanning carcasses for the New Zealand Meat Board, the models could be captured in real-time at an incredible resolution for the digital effects facility. This provided a very detailed and strict guideline for the form of the creatures, which the digital artists could then use in building up the characters for their animated performance.

Two Powers: More Than One Cook in the Kitchen

Randall Cook became interested in the movie business when he was seven years old, with the release of The 7th Voyage of Sinbad, and in seeing the other films of Ray Harryhausen. "That scarred me for life," he reminisces, fondly. Now, forty odd years later, Cook is the director of animation at Weta Digital.

Cook's background is one of traditional animation and training, having started out doing stop-motion work, and having studied acting and directing through the University of California, Los Angeles. In the 1970s, during the transition period from its earlier classics to its later revitalization, Cook worked as a storyboard artist and "gag man" for Walt Disney Studio. It was during this time that circumstances allowed him to be taken under the wing and tutored by one of the original Nine Old Men, Eric Larson. "I was fortunate enough to come into contact with some very clever, smart, talented old fellas," Cook says, "who knew a trick or two, and who were willing to teach me something." Cook worked alongside a group of young hopefuls, at the time, like Glen Keane, Randy Cartwright, Ron Clements, Ron Husband, Ed Gombert, Brad Bird and John Lasseter -- "people who have all gone on to something other than obscurity, as you may well know."

Randall Cook, Weta Digital's director of animation.

Cook met Peter Jackson after a career of doing animation on independent pictures, creating and staging special effects and action sequences, and producing "tussles with monsters," as he says, in films like The Gate and I, Madman, and "other films of even more meager budgets." Cook claims he was lucky that when he was making low budget films in his twenties, Jackson and Taylor were seeing those films when they were in their teens. As a result, when Jackson was old enough to start hiring people, he contacted Cook when he visited Los Angeles in 1992, to do the stop-motion animation for a film called Blubberhead.

"Mind you, I thought Peter was extremely bright and personable," Cook recalls, "but I wasn't prepared to move lock, stock and surface gauge to New Zealand to work on a film called Blubberhead or anything else."

Or so he thought.

When The Lord of the Rings came up, Lorna Cook, co-director of DreamWorks' Spirit: Stallion of the Cimarron, and Randall's spouse, encouraged him to call. In their earlier conversations, Jackson had made it very clear that he understood and appreciated the attention, effort and sacrifice that Cook put into his work, having surmounted and excelled within a low budget milieu. So, Cook called. Jackson offered him a job doing pre-vis for six months, not specifically animation. And that, of course, was three and a half years ago. Cook is still there.

In pulling together an animation department for Weta, Cook feels that his greatest contribution is being able to draw upon his acquaintanceship and friendship with Ray Harryhausen, Eric Larson and Tex Avery; in being able to pass along to the next generation of animators what his earlier mentors had taught him. For Cook, it has become a kind of "holy quest to make sure the right kind of animation gets into The Lord of the Rings."

Jackson is also a huge fan of King Kong and Ray Harryhausen. Cook explains, "It's natural for me to empathize with what Peter is talking about, because he and I have many of the same childhood touchstones to which we refer when discussing creature sequences." As such, there was a stylistic compatibility and passion to bring into the world of computer animation the same values of staging and feeling from the classic creature films of Willis O'Brien and Ray Harryhausen.

It turns out that Mr. Harryhausen was able to join The Lord of the Rings filmmakers, in December of last year, for the London premiere. It was a great treat and honor for Cook, Taylor and Jackson to have their boyhood idol enjoy and endorse their work.

Return: Weta's Creature Comfort

Perhaps one of the great successes of the movie is that the animation, and the special effects in general, was so well integrated into the story's telling, as opposed to being its raison d'etre.

For the water horses sequence, several images had to be combined to create the final plate in such a way that the live-action and visual effects would work as one including an original shot of the canyon, Ringwraiths on horseback in front of a blue screen, and CG water horses.

Richard Taylor remarks, "We believe that we have created a world that is not so much fantasy as mythological reality. So often in fantasy filmmaking the actor is required to wink at the audience, to ask them to buy into the film, because even the actors aren't taking it seriously. Well, here is a piece of filmmaking that supposedly should fit into that genre, where the actors are putting in A-grade performances in an A-grade environment. It's so imperative that the technology is used to create a seamless tapestry that is subtly woven together to put the backdrop into this incredible play; that at no time can the effects, the animation, the creatures grandstand themselves over and above the story that's being told through the performance."

Cook adds, "We were always trying to sublimate ourselves to the film. In creating these characters, the idea is always in being grand and theatrical enough to exist in Middle-earth, but with an eye towards being a part of the journey, rather than standing out from the needs of the story. A balance of naturalism and theatricality had to be maintained at all times to allow the characters to move effortlessly through Tolkien's world."

Much of the praise heaped upon the first film celebrates the technology that helped to create its expansive, epic scope. Weta's proprietary crowd behavioral simulation software, dubbed Massive, helps to build the huge battle scenes of which the prologue in The Fellowship of the Ring is but a taste. Every individual character in the crowd moves in response to the stimulus of its environment, such as terrain and obstacles, and the actions of other characters. (This differs from typical crowd, battle or spectacle scenes, wherein a small number of actors are filmed and then multiplied.) The Return of the King will see hundreds of thousands of these autonomous, intelligent agents in frame at the same time, making for a truly new cinematic experience.

The Massive technique is used to create the onslaught of Uruk-hai marching into battle. Multiple cameras are used in the motion-capture process to record the action of an Elf "massive agent" running up a ramp.

While the special effects are as much an accomplishment of the software programmers, who are oftentimes writing code late into the night, the state-of-the-art technology is only a starting point. Cook points out, "Massive, as an artificial intelligence program, is nevertheless only as good as the movement put into it, some of which is done by animation, yet much of which is done by motion-capture."

While the staff at Weta grew to almost 200 people during the production's first crunch time, the entire animation team numbered no more than twelve. The animators mixed, matched and switched off on the various scenes, with everyone contributing to everything.

The Cave Troll, Balrog and Watcher in the Water were animated through keyframing, and involved some of the biggest and most intense scenes. The Troll, in particular, required the time and talent of the whole team. Cook put on the motion-capture suit only to help in blocking out the sequence's camera work for Jackson's quick approval. But then the animators built up the scene one frame at a time, out of their raw imagination. The Troll sequence was animated by, in no particular order: Adam Valdez, Matt Logue, Andrew Calder, Carlos Rosas, Mary Victoria, Heather Knight, Doug Sheppeck, Melanie Cordan, Steven Hornby, Mike Stevens and Stephen Buckley. Their work is what we see on screen.

Animation of the Cave Troll was keyframed to the post match-moved plate of the hand-held camera work, and digital ceilings were tracked and seamed into the practical set. Like all the L.O.T.R. creatures, the Troll was constructed with a physiological sensibility, with working layers of muscles, fat and skin applied over an accurate skeleton.

Creating the animation for Gollum, however, was another matter. Gollum is based on the voice, pantomime and motion-capture performance of actor Andy Serkis, who does a wonderful job in helping to distill the emotional complexity of the character.

Jackson comments, "Weta developed vast amounts of code to create Gollum ... He is amazingly life-like, and we were able to give him a range of expressions, from the evil of Gollum to the sympathy of Smeagol."

In The Fellowship of the Ring, Gollum is brought to life using a combination of techniques. When the character is being tortured, for example, the keyframes for that scene were roto-matched to Serkis' performance; whereas the shots of Gollum in the Mines of Moria, as he peers at the Fellowhip through bars, is straightforward keyframing.

Hordes of Orcs descend upon the Fellowship in the Mines of Moria.  The Fellowship negotiates the treacherous stairs of Khazad-dum.

Other brief sequences from the first movie that impressed audiences were the stampede of water horses and the dragon fireworks. Cook remembers, "The dragon fireworks is very interesting, because we had to get the animators to animate in a way they weren't used to. It was very easy for the dragon to look like a wonderful flying dragon, whereas they had to make it look kind of mechanical. So, Doug Sheppeck, Mary Victoria and Stephen Buckley animated those scenes, and they had to work like hell to animate much more crudely than was their usual method."

Cook emphasizes the importance of animators honing their acting skills. "It should always be clear what the characters are thinking, that their intentions are readable," he says. The animators use dialogue tracks to work with the characters, sometimes written by themselves or by Jackson, so that they understand precisely what the characters are thinking or going through. In matching this internal dialogue with the eyes or other features of the character, though not the actual lips, a kind of "thought sync" emerges.

Cook explains, "It is vital that we have creatures that not only seem to be real, or naturalistic, but which also are charismatic, compelling, and that perform in a fashion that is dramatically valid."

For the several characters in the second and third films, some with digital fur, the animation will become more complex and involved. A few of the key characters include Gollum, the giant spideress Shelob and Treebeard (a walking, talking tree).

Cook says, "To anyone who has read the books, they know what a wonderful star turn Treebeard has the potential for giving, and I don't think they'll be disappointed. I'm very fond of him as a character. He's going to be unusual, entertaining and touching. Unlike our poor old Troll friend [in the first film], Treebeard has a lot to say for himself. He has a much greater range of emotions than the Troll, who spent much of the time angry and bewildered, and more than a little bit in pain."

The Show Goes On

A year ago, Weta had one rack of 32 processors. But it soon became clear that would not be enough to handle the backlogging queue of workstations needing to render their scenes. By the end of The Fellowship of the Ring, they had ramped up to almost 800 processors, and for the increasingly intensive and elaborate visual effects work for The Two Towers, it is expected they will be using about 1200 processors. Already, adjustments have been made to network switches and data distribution systems to ensure the technology will scale up again for the studio's growth peaks during the next several months, with the working storage space for the facility probably reaching about 30 terabytes for the second and third films.

The Balrog, a creature of shadow and fire, was realized by shooting practical licks of fire and smoke and then attaching these to a particle system. This gave the fully digital Balrog his mane of fire and cloak of black smoke. The hall of Balrog was created with the use of 1/14 scale miniatures.

Though about 90 percent of the company's systems are from SGI, there is a major development and research effort under way at the moment into running more Linux-based workstations. According to Weta's chief technology officer, Jon Labrie, they are looking to at least double the facility's number of installed Linux systems for the second film. The Linux machines are dual 1GHz Pentium III boxes running Red Hat 7.0.

Modeling and animation was done principally in Maya, using in-house proprietary muscle and skinning plug-ins. The team also used Mirai for some advanced facial animation setup and modeling, with Renderman handling the CG renders. Among the other tools was Shake handling the compositing, as well as Inferno, a Flame, and three Cineon seats.

So far, between $20-30 million has been spent on Weta's IT infrastructure.

In terms of recruiting artists and talent to New Zealand, it is less difficult now that the first movie is out. The challenge is to find people who believe passionately in the validity of the films as potential classics. Cook recalls, somewhat wryly, "When I learned that the guy who had made Heavenly Creatures ... and Braindead ... and who had written Blubberhead ... was going to do The Lord of the Rings, I thought, at the time, it had great potential for being a classic."

Cook and Adam Valdez, while recruiting, looked for people who had unusual abilities, and who had an approach to their work that was sympathetic with what The Lord of Rings filmmakers were trying to do. "Quirky, offbeat ... but work that was honest, and that didn't take the easy way out; that didn't take the knee-jerk, clichéd way out of a scene, but that had a greater breadth of invention than the norm," Cook explains. Gathering together artists from New Zealand, Europe and the United States, there are some high quality, high profile people coming on board. Presently, for the second film, the animation staff numbers at 15, though they are looking to bring on a few more people, because there is so much to do.

After the long haul of The Lord of the Rings, it is difficult to anticipate the future. Weta will likely continue to do service work for Peter Jackson and other top class directors, and there is always the possibility of creating intellectual properties of their own. Given the artists and technology they have assembled, there is ample opportunity to do high end effects, coupled with fantasy. While Weta hopes to continue to work on film projects they are proud of, their large facility and diverse range of people lend themselves equally well to servicing the creative industry as a whole whether that be educational children's television, interactive media, theme parks, museums, or any form of external display work.

For those financiers and producers who come to New Zealand, they realize, as Taylor observes, "That it's not just a clean, green beautiful shooting location, but there are highly enthusiastic and motivated crews, with the sophistication of any in the world." He adds, "There's still almost a youthful innocence to the industry that allows people to operate at a level of passion and enthusiasm that can run freely, because you're not caught up in the fashion of following preset rules."

In the world of film and entertainment, there are other forces at work than we may realize. And that indeed, dear Frodo, is encouraging news.

Greg Singer is an animation welfare advocate, eating in Los Angeles.