Dressed in a blue suit, hair tied back, Rea smiles at the young man as he approaches the desk in a Cambridge, Massachusetts, office. At the same time she is taking his measure.
Rea: Hello. Can I help you?
John: I'm looking to buy a place near MIT.
John: Tell me about it.
Rea: It's big . . . [She makes an expansive gesture with her arms, but noticing that John seems to want to cut in, she stops.]
John: Is there a garden?
Rea: Yes, there's a nice garden . . .
John is a real guy, but Rea (short for real estate agent) is a programmer's creation, computer code brought to life in three-dimensional, animated form. In computer talk she is an agent, a virbot (virtual robot), a bot, a graphical interface. This simulated two-way conversation takes place in the Media Lab at MIT. Rea is the creation of Professor Justine Cassell, a cognitive psychologist and linguistics expert. Rea is a virtual mannequin that seems semi-human.
Two digital video cameras trained on her client allow Rea to track John's hand and head positions while he talks into a microphone. John unconsciously becomes "entrained" to her--as he would with a real salesperson. She may look like what she is--an artificial person--but her actions and gestures are so convincingly human that John soon forgets she's only virtual. Her speech is syntactically correct, her gestures human. She is reading John and reacting to his words and gestures.
For centuries, scientists have been obsessed with the idea of creating artificial intelligence. With computers and the Internet, they are finally getting close to their goal.
The practical reason for all the activity now is to create "agents" that can serve as a bridge between real humans and the Internet. Some agents will work on voice recognition: You tell your car, "I'm in a nostalgic mood. Play me some Beatles." Familiar with your tastes, the agent does the rest, and shortly the appropriate sounds blare from your car stereo.
Other agents, in the form of graphical interfaces that elicit a reaction from you, will read your face and gestures as well as your voice and typed commands. Computer scientists and psychologists working together at such places as MIT and Stanford, are attempting to catalog the physiological and psychological ways people interact so as to program them in a virtual person. Rea is an early product.
The human-computer interface has been stuck on a keyboard, mouse and screen for some 15 years. "Today your Net experience is terribly limited," says Richard Sherlund, a software analyst at Goldman Sachs. "You can't even cut and paste in HTML. There's no flexibility in data input: no handwriting recognition, no voice recognition."
As part of its Internet initiative, which consumes half its $4.4 billion R&D budget, Microsoft is spending heavily to make it easier and more pleasant--more human--for people to interact with computers and the Internet.
Using statistical probability and decision-theory techniques that draw inferences from user behavior, these agents will know what kind of music you like and whether you prefer scenic back roads or just want to get there fast. Eugene Ball, a Microsoft senior scientist, says that these agents will be able to screen your e-mail and point to things of interest on the web.
While it will be at least three years before Microsoft uses agents in its operating system, other virbots have already begun to take posts on websites in sales and customer service. The animated sales representative, Kim, of Artificial Life in Boston, Massachusetts, sells cellular telephones online for the German telecom MobilCom. Kim wears a red suit and has dark brown, shoulder-length hair. Unlike Rea she can't "read" a sales prospect that approaches her on the Internet. But her pleasant voice, her winks, her smile, her preprogrammed gestures and humanoid appearance make her answers to your questions more engaging than mere typed answers would be. "It's a significant advantage that through Kim we can react directly to customers." says Thorsten Meier, a MobilCom executive. "Particularly when we're selling products that require a lot of explanation."
"Demand is really taking off right now," says Barbara Hayes-Roth, the CEO of Extempo, in Redwood City, California, which develops animated expert agents and natural language engines. Its software allows computers to engage in conversation using loosely scripted responses. Procter & Gamble hired Extempo to create a computer agent for its Mr. Clean character at www.mrclean.com. Ask Mr. Clean a simple question and the bald, muscular figure will smile broadly and say, "I wear an earring because it's fashionable." Crude as all this is, marketers find that these humanoids do make people more comfortable with computers.
Virtual Personalities, a company in Los Angeles that was founded by Michael Mauldin, creates virtual characters for customer service.
"Virbots are never rude, never take lunch breaks, never screw up, never ask for a raise," says Robert Rappaport, Virtual Personalities' CEO. "They provide a comfortable bridge between people and technology."
Artificial Life has created a virbot financial adviser, Ashton, who always wears a gray suit, blue shirt and checked tie. You can get much of the same information from a good financial portal, but Ashton makes the experience more personal.
What makes the virbots more human is this: Their expressions and gestures are coordinated with a database that assigns qualities to thousands of words. Thus the virbot smiles warmly when a visitor says, "Hello," and place his hands on his hips and grimaces when called "stupid." Robert Pantano, the chief financial officer of Artificial Life, says, "Small-talk capability gets people comfortable. People start talking about themselves, volunteering a lot of important stuff that's beneficial to companies--name, age, occupation. No offense to people mining cookie data, but we're taking online marketing to the next level. People find cookies intrusive. Not so our agents."
Virbots are expected to play a big role in the classroom as education moves to the web. They enhance the interactive nature of the medium. They can help keep students engaged and prevent their minds from wandering. "Studies show that children learn better when they're playing," says Hayes-Roth, who is also a Stanford professor of computer science. Extempo has developed an agent called a Web Learning Guide. Hayes-Roth describes it: "Like a good mom, the Web Learning Guide is sensitive to different learning styles. It creates a learner profile from online choices, performance and answers to explicit questions and then adapts its approach to that style."
Hayes-Roth points out that these techniques are useful not only with kids. She sees them playing a big role in corporate training (adult minds wander, too). "Not only would one-on-one guides be available to anyone, but it's also possible for each web educator to create one. It's a kind of self-publishing." A kind of animated second persona for every teacher.
Artificial Life has a virbot named Einstein who teaches--what else?--physics. If you've avoided answering a question for a long time, Professor Einstein taps on the chalkboard to get your attention. Do poorly on one of his quizzes and he will adjust the level of his next lecture.
Cassell, the creator of Rea, has spent 15 years observing videotaped conversations and hand motions. This led her to develop a formal model of the relationship between speech and gestures. She's interested in discovering the system of rules that underlies human gesture. Were you taught to keep your hands still while you were talking? Whoever taught you that probably did you a disservice.
Cassell has found that many gestures of hand, shoulder or face frequently help convey information. While Rea gazes intently into a visitor's eyes, five processors record the visitor's gestures and inflections. Does the client say, "Oh, this bathroom is beautiful. I love blue tiles," with a gesture on "blue"? Rea catches that movement and replies, "Yes, blue is a lovely choice for a bathroom."
But if the client says, "Oh this bathroom is beautiful. I love blue tile," Rea replies, "Yes, tile is so easy to take care of." Identical sentences, but unless the computer can read the gesture, it misses half the cues.
Rea is still years away from your desktop. There's not enough bandwidth yet to accommodate all of the video and audio data she needs. When the bandwidth is here, Rea and Einstein and Ashton will be ready. When they go into their full acts, the computer and the Internet will be closer to realizing their full potential.
There's a downside. Artificial Life's Luci is a sultry redhead who occasionally winks at visitors and keeps a log of every conversation she has. We asked Luci what she thought of characters that play manipulative roles and violate users' privacy.
Luci smiled. Then she replied: "I'm not going to lie to you. But I may not be as direct as you'd like."