[Figure 9.64] Still from an early test done for Madame Tutli-Putli. (Courtesy of Jason Walker.)

I asked Chris and Maciek to shoot some simple moves of a test puppet with blank eyes and did a test with filmed footage of actress Laurie Walker. Three basic steps were required to try to make it work: film the actress, track the puppet, and stabilize the footage of the eyes to stick onto the puppet face. Luckily, after about a month of working on it in my spare time, the test worked (Figure 9.64).

The directors were blown away by it, but I told them if they take this farther, they needed to make sure the puppet didn’t move around very much. I went away, and they got approval from the National Film Board to make the film. Coming back a month later, I saw their footage of all this puppet movement, with flashing lights and shadows from the moving train scenes. I started thinking, “I hope my method will work for this!”

[Figure 9.65] Images of Laurie Walker’s eyes and head movements being filmed in live action. (Courtesy of Jason Walker.)

Based on the timing and movement of the puppet animation, on the live-action set, we worked together to simulate the light flashes and direct Laurie to mimic the head movements. She had make-up and tracking markers applied to her face for matching to the puppet, and she was told exactly how to move her head and directed on the acting and emotion of her eyes (Figure 9.65). I tried to keep her on track with the correct movements and orchestrated the lighting, and there was no room for error as scenes became more complicated.

[Figure 9.66] Still images of Madame Tutli-Putli’s puppet animation, with the Wunderbar timeline. (Courtesy of Jason Walker.)

When I had the final eye take, I would bring it into the computer and try anything to make it work. I had created a timeline chart in After Effects and nicknamed it the “Wunderbar” (Figure 9.66). Once I had been given the puppet footage, I would analyze what was a head move and what was a camera move, and indicate each on this timeline as a different color. This way, I could see a separation between what was a move and what wasn’t. Also, for moments when she would encounter a light or a shadow, the Wunderbar would record what kind of light it was and how long it lasted.

[Figure 9.67] A progression of steps showing the puppet’s tracking markers, masks, and composited eyes. (Courtesy of Jason Walker.)

On the stop-motion set, there were tracking dots for the eyes built onto the face of the puppet. You would think that would be helpful for all the stabilization, but the dots were only used to track the puppet so that I could adhere a mask layer to the face. I had the eye layer separate from a layer of masks that cut the eye out (Figure 9.67).

However, when it came time to place the eye footage onto the puppet, there was no way to do it except by hand. That was the most intricate part; the computer helps you organize your layers, cut masks, and feather the edges, but the computer has no idea what an eye is, and it has no idea of the subtlety of a human eye in the area of a human face. When it comes to visual effects, it’s on one level to make it flawless, but the other level is to make your brain convinced in a way that you don’t have to think about. A bad composite is when a scene seems to look right, but the brain tells you it doesn’t. When it comes to human eyes, there is absolutely no room for error. When placing the eye onto the puppet in After Effects, if the eye was off by even a fraction of a pixel, it wouldn’t work.

So, I developed a system where for every frame, I would need to zoom all the way in, use the arrow keys to move the eye up and down, and then zoom out and see if there was any independent movement. This also had to be done for scale and rotation of every frame. Often, I would zoom in, move it over one pixel, and then zoom out, and it would be too far over. Then I wondered, “How can that be, if it’s only a pixel? How can I move it less than a pixel?” It started to become insanity at this point, but my solution was this: Let’s say my eye was in the right place on frame 10, and I move it over one pixel for frame 11, but it was too far. What I would do is put a point there, but then drag that point over to frame 12 so that frame 11 was right in the middle. To explain this further, let’s say you were on one side of a fence and you could only jump to the other side of the fence, but you wanted to be on the fence. You would build a wall so that when you jump over the fence, you would hit the wall before you could land on the other side, but at least you would land on the fence. That was the only way to make the eyes convincing, and the toughest part was this method of sub-pixel positioning so that it always looked like it was on the puppet.

Another point to make is that this technique has sometimes been described as simply adding live-action eyes to puppets, so people think it’s not animation. This is partly correct, in that they are real eyes, but it’s not merely live action. When I paint portraits of children, they don’t sit still very long, so I shoot video of them, and afterward I can search until I get that one instance of the child’s face that is right for the painting. The same technique applied here, where I would film the eyes and the actress going slower than the puppet, matching accuracy of the movements, but not timing of the movement. Often, I would film the actress moving at least six times, take these separate takes, and join the eyes together. I would bring in these sequences, which would add up to many hundreds of frames, but inside the scene of the puppet there might be only 100 frames.

Then, it was a matter of selectively going through each frame of the eyes, using the time-remapping feature of After Effects, and sliding through the frames one at a time until you find the one frame that works for that frame of puppet. Essentially, it’s a re-animation of video stills—people may think it’s not animation, but it is! I have to animate the character that’s coming out of the eyes, and part of that is measuring how much she reacts to things by how many extra frames you have her looking there. You have the body language of the puppet and great performance of the actress, but there is also a third level where you can change the acting. Going back to Lord of the Rings again, it was noted that if the actor playing Gandalf didn’t look concerned enough, for example, they would use a subtle computer mesh to change his expression that much more.

I have ideas for some similar animation techniques I want to try with the entire face, but I will only do that with the right team of people. I often get approached to help other people with these kinds of techniques, and many people ask me if there is any new technology developed in last 3 years that will help, like 3D scanning. None of that really helps because you still have to manually position a human eye one frame at a time onto the head, and only your brain will know if it looks right. Your computer is not going to understand human emotion—only your brain can do that. Seeing human eyes on a stop-motion puppet in Madame Tutli-Putli is something we’ve never seen before, and the effect of the film comes down to the fact that it looks like the eyes are there. Anything beyond that is a failure because it’s the eyes, and everyone in the world is an expert on this. If there is something wrong with the eyes, you know it right away. There is a quote where someone said, “If you don’t believe eyes hold the human soul, then take a picture of someone you love and stab it in the eyes with a pair of scissors. I bet you can’t do it.” That’s the power of eyes.

For more information on the film Madame Tutli-Putli itself, visit http://films.nfb.ca/madame-tutli-putli/.

Ken A. Priebe has a BFA from University of Michigan and a classical animation certificate from Vancouver Institute of Media Arts (VanArts). He teaches stop-motion animation courses at VanArts and the Academy of Art University Cybercampus and has worked as a 2D animator on several games and short films for Thunderbean Animation, Bigfott Studios, and his own independent projects. Ken has participated as a speaker and volunteer for the Vancouver ACM SIGGRAPH Chapter and is founder of the Breath of Life Animation Festival, an annual outreach event of animation workshops for children and their families. He is also a filmmaker, writer, puppeteer, animation historian, and author of the book The Art of Stop-Motion Animation. Ken lives near Vancouver, BC, with his graphic-artist wife Janet and their two children, Ariel and Xander.