<em>The Beatles: Rock Band</em> -- A New Revolution in Gaming

Harmonix partnered with Curious Pictures on a new Maya-based facial system that allowed John, Paul, George and Ringo to appear more human yet still iconic and stylized. All images courtesy of MTV Games.

Watch the trailers for The Beatles: Rock Band. [1]

The Beatles: Rock Band videogame finally arrives today (from MTV Games), allowing you to pick up the guitar, bass, mic or drums and experience 45 songs from the Fab Four's renowned music catalog through gameplay that takes you on a journey through the legacy and evolution of their legendary career. From the early touring days in 1963 Liverpool at The Cavern Club to The Ed Sullivan Theater to various, psychedelic "dreamscapes," to the immortal, final concert on the Apple Corps rooftop, you can now get closer to The Beatles.

Meanwhile, the Beatles catalog is also being reissued today (from Apple Corps Ltd. and EMI Music): carefully re-mastered in stereo and mono (available only in a collectible box set) using the original analog master tapes. De-noising technology was kept to a minimum, as was overall limiting to boost volume, and only on the stereo recordings.

Josh Randall, Harmonix's game lead, explained how song selection was decided for Rock Band: "We began by brainstorming with Giles Martin [music producer and son of Beatles producer George Martin] from Abbey Road Studios on a list of songs that would be fun to play in our game, which we felt was an important place to start. We wanted the game to span the entire performing and recording history of The Beatles, so we tried to select songs, outfits, eras and venues that would represent key milestones in their amazing career. I think in the end we came up with a great selection of popular favorites, and deep cuts...there should be something in this game for everyone!"

The Beatles: Rock Band was developed by MTV Games' Cambridge, Mass.-based game producer Harmonix (Rock Band, Guitar Hero), and I spoke with Art Director Ryan Lesser [2] about the thrill and challenge of raising the franchise bar for more emotive character animation and sophisticated visuals.

Bill Desowitz: So this was no ordinary Rock Band project, was it?

Ryan Lesser: Yeah, it was a pretty crazy project this time around.

BD: So, talk about wrapping yourself around The Beatles visually -- the character animation as well as the various environments.

RL: One of the first things that we started working on was their likenesses. Very early on we did a demo for the sake of the shareholders and Apple to make sure everyone was onboard with the project, and the concept was non-realistic, iconic imagery for The Beatles. We didn't want to find ourselves in the typical area of the Uncanny Valley in depicting humans realistically -- and probably not doing them justice. So, we were looking for a style that was very endearing and charming in the way that those guys were and yet was still functional and fully animatable and did everything we needed it to do. So we actually didn't do a ton of early drawing -- we did some. We really started working with ZBrush as a sketching tool and we went in very wild directions: cartoony, less cartoony, in squares, more organic and smooth. And what we wound up with is in the same universe as the other Rock Band character stylization, but not the same. You can see differences.

And this demo, which is just the guys frozen in time, has a little of a bullet-time thing around them in the stadium. They had all the right gear, the lighting was appropriate, the outfits were right, their hair was right. It was the first time that anyone saw these characters in this style. And when we showed it, it was received pretty warmly. Of course, it wasn't perfect -- it wasn't presented as perfect. But it was a lot easier to show how the game was going to look like.

The dreamscapes allowed for experimentation and psychedelic stylization, while the post process suite provided color correction, mirroring, kaleidoscopes and frame flicker.

BD: How did you accomplish the appropriate likenesses and movement?

RL: We had a lot of issues with getting these guys to emote in the way that they did in life in the band. And that was a big deal to us. Everyone knows what these guys look like. I think that a lot of what they've done has a lot of immortality to it. These songs and these images and videos that people have come to love are going to last forever, and we didn't want people to think that this would be a piece of media that would be fleeting; we wanted to participate in the continuing evolution of The Beatles imagery. We wanted to make our own impression as well and add to this really amazing and diverse catalog of theirs, so we actually spread out over a couple of different art styles. There's the Cinematic [3], which has one art style that’s' related to the game but is not exactly the game style, and then we have the gameplay stuff, so we had to think through this multiple times. And we didn't take anything from existing media; we either invented it ourselves or revved it and put it through our filter.

BD: What pipeline did you use?

RL: ZBrush for doing rough 3D sketches, and the final work was done in 3ds Max [4], which is getting better and better every year, and ZBrush is about to release GoZ, which we're pretty excited about -- it's a one button back and forth between the two programs. We have Milo, which is our in-house rendering engine and is developing into a full level editor with 3ds Max. We use a lot of MotionBuilder [5] as well for body animation and Mocap. And we have a suite of post processes that we use in Milo to alter the final image, and it allows us to do really crazy stuff like color correction and mirroring and kaleidoscopes and frame flicker.

BD: And what did you do that was new for the facial animation?

RL: For this particular project, we wanted a lot of careful detail in the faces and so we developed a new technique with some our animators and with Curious Pictures. They partner with us sometimes: they do Mocap and other little technical things for us. And they worked with us on a new face system, something we've never had before, just to get real subtlety in their faces and have some little quirks. Unlike Rock Band or any of our older games, we had people that are recognizable, not only visage that's familiar to viewers, but also the musculature and skeleton and everything that happens on a person's face. People know John's smile and Paul's eyebrows. There are little things that we had to nail and still do it stylistically and still make it very obvious. So we built a new tool, which seems crazy how many different tools we used to make the game. But at this point, we use what we can to get the job done. And with some of the more technical animators and programmers here and at Curious were interested in using Maya for this particular face rig, so we incorporated Maya [6] for the first time ever in our workflow.

The early depiction of George was deemed too much like a superhero, so the Harmonix team pulled back on over emphasizing his cheek bones, chin and jaw line.

BD: And how did that work out?

RL: Using Maya [6] was fine. It's a good tool but there have been some regrets in introducing more and more software. We're trying to pare down a little bit, but we would never do it to the detriment of the project. There is a tax with every tool you add to the pipeline. 

BD: So, how did the new facial system work?

RL: It was a mixture of MoCap and keyframe and had a unique skeleton that allowed us to deal with the whole face and entire set of emotions.

BD: Tell me about working on some of the more imaginative sequences.

RL: As far as sequences and spaces and environments and such, we split them into two general groups: historical venues and dreamscapes. And historical venues are really fun because we get to riff on places that exist in the world and don't anymore that The Beatles actually performed in. This is really fun doing recreations and again putting it through our style filter.

BD: And what kind of input did you have from Paul and Ringo and the surviving family members of John and George?

RL: They were there and we weren't, so it was really great to get input from them about, for example, The Cavern Club. We have very limited media available. But to be able to hear from people at Apple and the folks in the band about atmosphere, that it was very smoky and dark and very close quarters, was [essential] because when we first started, we didn't have that.

And then the second group is the dreamscapes where we really got to experiment and spread out with imagery that hasn't existed before. And we went in a lot of different directions, like "Walrus," for example. It was very trippy and embraced the psychedelic nature of the later recordings, whereas "U.S.S.R." is very graphic with simplified color schemes related to propaganda art back then but also modernized and fluid. This is sort of a modern interpretation of their work.

BD: What kind of feedback did you get from The Beatles when they saw it?

RL: This one went pretty smoothly. They really liked it. We didn't get a lot of feedback on this one for changes, as far as I recall. They were pretty happy with what they saw, for the most part, and would only ask for subtle changes.

BD: What was some of the more interesting feedback you received from The Beatles or surviving family members?

RL: I guess the two that I recall are when Josh Randall, Chris Foster (the lead designer) and I went to Apple for the first time with images of George. And the style at that moment was very heroic, and I love George Harrison and the striking way he looks, and we made him very superheroesque, and dug in his cheek bones and chin and jaw line. And those guys at Apple were very critical in a very positive way. A lot of us have gone to art school and are used to having strong critiques, and these guys did an excellent job of honing in on the things that make these faces right. We came back really knowing we had to dig in again.

And then Yoko made a visit at one point here at the office to check out everything we were working on, and one of our favorite sequences was The Rooftop. And, again, just like The Cavern Club comments I made earlier, we weren't at The Rooftop, and there were all these subtle things that Yoko was amazing at picking up on, like the way the way the wind was blowing and how John had a very specific way of standing and how strong and bigger than life he was. It was kind of mind blowing for someone of her experience to sit down with my animator or my modeler and work one on one, artist to artist.

For the famous Rooftop concert, Yoko Ono provided invaluable advice about the way the wind was blowing that day and how John stood confidently and heroically.

BD: And Paul, who is a big animation fan?

RL: I, unfortunately, never got to meet Paul, but Josh Randall spent time with him on multiple occasions. And Josh says that he was incredibly fun to be around and he was really into the animation, just as you said. He really liked pretty much everything we did and would make comments. We would show him a song and he would pick up on the animation or on the editing and would be like, "Oh, on this particular part, I was playing that part, John was playing this other instrument or Ringo wasn't there that day." So we were able to switch instruments and have it be correct, and we built the whole game that way, allowing guys in the band to switch instruments, which we have never done in a game before. It's another facet of the game that shows The Beatles putting such cross-instrumental effort into all of their albums.

BD: Have they played the game?

RL: Oh, yeah. Josh played with Ringo and Paul and this was three-quarters of the way through the game and they were having a lot of fun and poking fun at Josh a little bit for messing up Ringo's drum part, which was pretty intimidating to have The Beatles sitting there watching you fake play their songs. But after a really positive critique, Paul left the room singing, "It's getting better all the time."

 Bill Desowitz is senior editor of AWN & VFXWorld.