A new and growing application of animation is serious games -- videogames used for training soldiers, policemen, firefighters, medical professionals, retail clerks and a host of others. The market for such games has risen in the last five years from essentially nil to a few hundred million dollars annually. One of the most successful serious games is Tactical Iraqi, a game designed to teach Arabic to soldiers deployed in Iraq, and developed by a team from ISI (the Information Sciences Institute, part of the University of Southern California) headed by Dr. Lewis Johnson. The methods used to develop this game shed light on what really works in successful serious game production -- much of which is very different from techniques used for entertainment games.
Learning a language the conventional way tends to be painful and arduous -- just remember the boring vocabulary drills from high school or college language classes. It also requires a classroom and a teacher, something hard to find in Iraq, thousands of miles from the nearest Army language school. Arabic is an especially difficult language for English speakers to learn, according to Dr. Johnson, who is a linguistics and artificial intelligence (AI) expert, as it contains sounds that are hard to distinguish from each other, and dialects differ considerably by region. "People who learn literary Arabic in school often have a hard time on the street," he said. Unfortunately, "on the street" is exactly where our soldiers in the Middle East need to use it -- trying to pull out a dictionary during an altercation on a crowded boulevard in Baghdad is probably not a wise choice.
The game is not just about learning the right words, noted Dr. Johnson. It also involves a huge amount of nonverbal interaction; scientists claim that over 70% of communication comes from what some call "body language." In tense situations such as those in Iraq, nonverbal messages may be just as important as the spoken words, according to Dr. Ralph Chatham, the sponsor of this project from DARPA (the Defense Advanced Research Projects Agency, the cutting-edge Pentagon agency that created the Internet, online gaming and modern satellites, among other things). Dr. Chatham noted that soldiers that first went to villages in Afghanistan were astonished to discover that they not only did not understand a word being spoken, but could not interpret people's hand and facial gestures. As "Sergeant Smith," a role you assume in Tactical Iraqi, you learn that when you start a conversation with someone important, you not only say, "Asalama Aleikum," but also cross your hand over your heart and give a slight bow, as a sign of respect.
The Game Play You start with the first of three components of the game, called the Skill Builder, an interactive area where you learn basic Arabic vocabulary. You wear a headset with earphones and a mike, so that you can hear the words, and so that your laptop can hear you pronounce them. You get immediate feedback, in friendly form -- if you mispronounce a phrase, the computer gently urges you to try it again. Throughout, the game is easy on you, with different, more tolerant feedback for beginners, so you don't get discouraged -- it gets harder when you move up to "expert." After you're up to snuff on Skill Builder, you enter the arcade, where you practice select words and phrases, such as colors, military ranks or how to get directions -- and score points as you might in a pinball game, to make the process fun and competitive. You're now ready to enter the Mission Game, the actual 3D scenario you act out in the mean streets of Iraq.
You are Sergeant Smith, an Army NCO about to deploy to Iraq. You have about 80 hours to learn the language well enough to interact with Iraqis, sometimes in tense situations.
As Sergeant Smith, you find yourself in Baghdad. Your first mission is to find and meet with the local honcho. You walk over to a sidewalk café, and approach an Iraqi gentleman (you know better than to approach an Iraqi lady). You greet him, with the appropriate gestures. You see positive symbols rise above the man, game-induced feedback that you're scoring "trust" points, what some people might call "social capital." You ask what area you are in, and who's in charge. You see the man start to scowl, and the trust symbols turning negative. Oh, oh, something's wrong. Tension mounts. The air becomes so thick you could cut it with a knife. You sense the man is going to say something to you, and it will be nasty. You also sense that there are many more of them than there are of you. But you have two things going for you, fortunately -- you're still at the "beginner" level, and you have your trusty (automated) assistant, a fellow NCO, standing next to you, who helpfully suggests that you might want to introduce yourself first. Ahhh, that was how you blew it.
You apologize deeply, overdo it a little bit with the bowing and introduce yourself with your name, and unit and what you are doing there -- you're part of a rebuilding team, trying to repair a local first aid clinic. You thank the man for any help he may be able to give you. You start thinking about how all this thanking and apologizing and bowing is really, really different from the typical entertainment-oriented videogame. But there is hope -- the man starts to relax and smile, the tension loosens a little and the trust symbols turn green again and he replies that he understands what you are doing. You ask him who the local chieftain is, and how you could meet him and the man starts to help you. You are not through the woods yet, but you have passed the first test.
You still have many other missions ahead of you, including attending a local dinner, manning a checkpoint, attending a wedding, and so on. Each will build your repertoire further. Each will test your understanding of local culture, your vocabulary, and your pronunciation - the game monitors your speech throughout, and a mispronounced word may bring you the beginning of a dust-up. Gradually, confidence increases, you move up to "expert" and you start to respond to situations instinctively, without having to carefully redact everything you say.
The Game Development "We don't have the luxury of picking our audience, like a commercial game, which can afford to lose a lot of potential players," noted Dr. Johnson. "We didn't have that kind of leeway -- we couldn't develop a game that half of the soldiers couldn't understand. We had to make sure that the game was playable by just about every war fighter that would need to use it. The game should not irritate the player, and make him give up. It should remain a fun challenge -- what Jim Gee (a gaming guru) calls 'pleasantly frustrating.'"
Because serious games typically have very limited development times and budgets (compared to major commercial games, which typically run more than $10 million and take two to four years to produce), they tend to be created using a lot of existing middleware, including game engines. This was the case with Tactical Iraqi, which uses the Unreal Tournament 2003 videogame engine. Although the original Unreal is an FPS (First Person Shooter) game, its adaptation lets you move your avatar (your game character) around richly detailed streets and meet well-rendered game characters -- but does not let you shoot them if they do not agree with you. Although Tactical Iraqi's development time was about normal for serious games (a little more than a year), its budget, at around $7.2 million, was higher than the normal $1 million range for such games -- but so were the requirements for the game, and the features that had to be developed.
One major game element that had to be created was the AI for the assistant and the other "smart" NPCs (Non Playing Characters) in the game. The reason for this is that the other characters are not "manned" -- they are not activated by other players, but by the AI software, and must automatically react appropriately to the actions of the player. Whereas other parts of the game may use some short cuts (the number of cars on the street may be limited, for instance, and their license plates may not be totally correct), the accuracy of the AI had to be exactly right, else the game risked misinforming the player about what to expect in real life (what is termed "false training") -- which could have disastrous consequences.
Whereas the development of entertainment-oriented videogames is relatively well understood, and can be found described in many textbooks and instruction manuals, the development of serious games appears to still be as much of an art as it is a science. A serious game needs to toe a fine line between being a traditional training tool (and thus prone to being boring and overly academic) and an entertainment venue (and thus irrelevant for learning anything about the task at hand). On the one hand, academic developers tend to, in the words of Professor Linda Polin of Pepperdine University, an expert in GBL (Game Based Learning), "Suck all the fun out of a game." On the other hand, most game developers like to make a game really challenging and cutting edge -- which is great for hard core gamers, but would leave most average students behind in the dust.
Both the speech and the non-verbal communication are critical game qualities, according to Dr. Hannes Vilhjalmsson, a scientist at USC who specializes in AI, and producing the correct artificial intelligence for all of the characters was a major part of Tactical Iraqi's development. The characters in the game don't just talk -- they may also take certain actions, such as shouting at you and advancing towards you if you give off the wrong cues, or helping you and taking you to where you need to go if you do things correctly. Whereas all commercial games use some level of AI, no entertainment game would normally be called upon to react to subtle speech patterns or body movements on the part of the player. But this is the case in Tactical Iraqi, where the computer-generated characters shrug, wink, nod or wave at you -- while following your actions with their eyes as you move around. The USC-developed software module that enables this level of fine movement, "pulling the strings" on the characters, is called Social Puppet.
Because an instructor plays a crucial role in a serious game of this type, it was decided that a surrogate for such a person had to be provided -- a real person could not be used, since the game is designed to be played with only the player present (a future version may be online, with multiple players). A robotic assistant, sometimes called an "intelligent agent," was developed to follow the player around in each scenario, and offer helpful hints when the player makes mistakes.
Another way that development for this game was different from that of the typical entertainment game was in the amount and type of prototyping and feedback. A normal game is designed and built almost to completion before testing with actual gamers starts, and testing the play with gamers outside of the company usually does not happen until the game is already in alpha or beta stage, almost ready to ship. By contrast, Tactical Iraqi was developed as a very rough version (called a "rapid prototype") and tested by outsiders almost at the outset, in order to get feedback on the authenticity of both the content (the vocabulary and movement) and context (the scenery and objects) of the game.
"We felt it unwise to employ real users at the beginning, because they might not be able to tolerate the gaps and mistakes in the game, and would get frustrated," said Dr. Johnson. "Instead, we used a 'User Representative,' someone who can fill in the gaps and see the 'big picture' of the game design." The first round of such tests was conducted by a cadet from West Point -- he was not a perfect representation of a future user (he already could speak perfect Arabic), but he was close enough (he was in the military, and could "put himself into the place of" a novice player), and had the expertise to recognize shortfalls in the language learning part of the game. The feedback from the cadet was incorporated, and the game was then tested on seven people who worked for the Institute, but had not been part of the game development team (they were thus "outsiders.") The game sessions were videotaped, as were interviews after each session. Further changes were made. This was iterated several times. Animation was largely conducted by USC students (the university has a large game development department, with extensive courses in animation, programming and graphics) and ISI staffers, with some help from outside sources.
After improvements, the game was tried on 21 actual users. Because the game has two major components (the basic vocabulary drill and then the 3D game itself), this group was split into segments, so each component could be tested. One result was that the after-game test of the player was moved from being a separate exercise to being within the game itself, as an in-game assessment tool. The game was tested again, with new groups of users. It was found useful to create practice sessions for the game, rather than have users just plunge in. Finally, the game was tested with 50 U.S. Marines, and compared with conventional teaching. The response from the troops was enthusiastic -- most young soldiers have grown up with videogames, and much prefer playing them to learn something than reading boring Field Manuals. After more modifications, the game was tested in its final version for the very first time. It has since been modified for different users -- U.S. Marines, for instance, have a version with different uniforms and procedures than do U.S. Army soldiers.
All of this testing and feedback was absolutely necessary, according to Dr. Johnson, to assure the game was what the users needed, and the designers did not proceed blindly toward dead-ends. Conducting such iterative short tests is not only orthogonal to what commercial game developers normally do, it also differs greatly from traditional academic testing of training material, which tends to be much more rigorous, with much larger groups of test subjects. "Working with larger groups of testers might have yielded somewhat more insight, but that takes too long, and could not have been coupled to the development process," noted Dr. Johnson.
The Future The game has proved out a great deal of the research in memory formation, which suggests that people remember things much, much better when they are emotionally involved in learning -- when they are in a heightened state of mental engagement with their surroundings. I can vouch for this personally, having played the game and missed several cues, only to find myself being accused of being a CIA spy, and having people starting to rise up from adjoining tables staring at me -- I was definitely fully present at that point.
DARPA has been very happy with the success of Tactical Iraqi, which is now being used by thousands of troops in Iraq. In fact, this project received a DARPA "Significant Technical Achievement" award in 2005, the equivalent of getting an Oscar from the Pentagon, and all the more impressive when one considers that the agency (which has a budget exceeding $3 billion) handed out only two such awards that year.
Further versions of the game are being produced, both for different users (U.S. Marine Corps, Army and civilians) and other Middle Eastern languages, such as Pashto, for Afghanistan (presumably, the graphics for that geographical area could be called "Afghanistanimation.") The intent is that each game will deliver what Dr. Chatham calls "tactical" language skills, that is, linguistic skills for specific purposes, rather than generalized "cover the waterfront" knowledge of a language. This type of training is sometimes called JITT (Just In Time Training), meant to be delivered right before it is needed, rather than traditional training, which is delivered in a classroom and then (hopefully) recalled many months or years later. The demand for JITT, especially in the form of videogame-based learning, is expected to grow rapidly, with delivery platforms, including both laptops and handheld devices.
Other languages, such as French and German, are under consideration (you could play a game and learn the important parts of the language on the plane to Paris, for instance), and online versions of all of the games are planned. A spin-off private company, Tactical Language Training, LLC, has been formed to market and promote the games to both the government and the general public, as well as to provide training, support and customized content development help.
For a closing note, I can't think of anything more appropriate than what I was recently told by Dr. Anthony Tether, the director of DARPA. "The system's success can be summarized by one soldier's words: 'I learned more in one day with this game than I learned in my whole tour in Iraq.'" Dr. Tether added, "This will save lives."
Christopher Harz is an executive consultant for new media. He has produced videogames for films such as Spawn, The Fifth Element, Titanic and Lost in Space. As Perceptronics svp of program development, Harz helped build the first massively multiplayer online game worlds, including the $240 million 3-D SIMNET. He worked on C3I, combat robots and war gaming at the RAND Corp., the military think tank.
Links:
[1] http://www.awn.com/imagepicker/image/11945
[2] http://www.awn.com/imagepicker/image/11946
[3] http://www.awn.com/imagepicker/image/11947
[4] http://www.awn.com/imagepicker/image/11948
[5] http://www.awn.com/imagepicker/image/11949
[6] http://www.awn.com/imagepicker/image/11950
[7] http://www.awn.com/imagepicker/image/11951
