Former Pixar alums create an animated, AI and speech recognition-based interactive iPad app for kids.
Several years ago, business partners Oren Jacob and Martin Reddy came up with an idea to use speech recognition to drive character-based interactive entertainment, starting a company, ToyTalk, to develop needed interactive tools and technology. Enter Bobby Podesta. Having left Pixar after 14 years, most recently as a supervising animator on Toy Story 3, he was looking for new challenges in storytelling and entertainment development. After meeting Oren and Martin, he became enthralled with their working prototype, and immediately saw great potential for their new interactive entertainment engine. Podesta joined ToyTalk soon thereafter and set about directing their first show. With $16 million in venture capital and a top team of creative types, ToyTalk just last week launched The Winston Show, their first commercial entertainment application release. The app is now available free of charge through the Apple App Store.
Available for free on the iPad and geared for kids 5-10, The Winston Show is an interactive conversation-driven talk show that moves across topics and activities in real-time based upon the user’s verbal input. Speech recognition and artificial intelligence drive the two main characters, Winston and his sidekick Ellington, around their CG set and the various activities at their disposal, all based on the unique interaction with the user.
Winston and his sidekick Ellington.
According to Podesta, the goal has always been to make animated characters that interacted in real-time with their audience. “The iPad isn’t just a one-way experience like a TV. It’s a two-way communication device. Built in camera, microphone, internet connection. As an animator, what could be more rewarding than creating characters that interact with and entertain you right then and there as you talk to them in real-time?”
The app contains no pre-rendered animation. Everything is done dynamically on the fly. To make this happen, ToyTalk built their own proprietary technology called PullString, which integrates dialogue with sketches, artwork and animation for two-way AI-driven conversations. It’s a sophisticated authoring tool that triggers animation, sound effects and other character behaviors. It is even designed to handle subtleties in the interaction, such as defining how the characters react to being ignored, or when a user changes the subject. But at its core, the technology enables creativity by allowing artists to develop shows like The Winston Show without deep technical or programming knowledge. Real-time interactive conversations with animated characters opens up a whole world or possibilities. The underlying technology they’ve developed provides another creative tool for artists to work with.
With six hours of content available in the initial app, pre-rendered animation wasn’t an option. Instead, the system sends out information to drive the animation in the form of splines, which feed CG models within the device, providing a real-time animated experience. Models and environments are animated in Maya and other digital tools.
The main benefit of this computing model is, of course, is that you can continually feed animation to the device. It’s not bound by a preset library. As Podesta explained, “Because we’re listening to you and want to respond back in a believable way, we need to harness computing power that is not on the device. The iPad is a wonderful platform. But because we have this huge amount of computing power in the cloud, we can do a lot more than you would normally be able to do in an app.” Their technology works fast enough to make the interactive processing transparent to the user.
Interactive applications, especially for kids, probably number in the billions. Every personal computing platform released since the Commodore VIC 20 has boasted a wide range of interactive games and educational programs for kids. So what makes this program unique? As Podesta described, instead of “controlling” characters, activities and environment, you’re “interacting” with them. You never know how they’re going to react or what they’re going to say. “These characters react to things based on how your conversation is going. That leads to having a believable relationship. That’s what makes all of us real. We have opinions. We react to things. Having a conversation with Winston is very different from making him walk across the screen and punch something. This is not a game. It’s closer to a live ‘show’ than a pre-scripted linear format. Users interact with engaging characters, engaging worlds and engaging conversation.”
Podesta continued, “Computers don’t make great characters. Artists do. This is just a different way of storytelling.” One major point Bobby stressed was that unlike traditional artificial intelligence programming, they weren’t trying to create something real. They were trying to create something believable. Plus, they’re doing it with entertainment. “I’m not trying to fool you into thinking there is a human on the other side. I’m trying to entertain you by your believing in the character that is being presented. That’s a very big difference.”
--
Dan Sarto is editor-in-chief and publisher of Animation World Network.
Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.
