Multimodal promises

MULTIMODAL technology offers the ability to use a combination of voice and visual interfaces in the same application at the same time. I provide this definition for those of you who have an automatic hype sensor that creates an impenetrable firewall between you and the hype the moment you hear a vendor repeat the same phrase, thereby blocking out any knowledge of this promising UI.

Not only is the term overused and the technology underdeployed, but by bickering over who has the best multimodal development environment, large high-tech companies are behaving worse than George Costanza’s parents.

I would like to put into perspective what the real benefits of multimodal technology will be — once IBM, Microsoft, et al., stop fighting and start producing — and to examine what some of the major challenges could be to multimodal’s deployment.

Steve Chambers, vice president of worldwide marketing at Boston-based SpeechWorks, laid out what some SpeechWorks customers want from multimodal technology.

A large beverage distributor would like to be able to give its field force a handheld or tablet PC so that when a technician is servicing, say, a clogged soda machine, he or she can use multimodal’s voice capabilities to ask for a schematic of the machine or, someday, use its visual interface to view a full-motion video to walk him or her through the repair. “Right now they have to schlep a huge manual or take the schematics on CD, which is very expensive,” Chambers said.

An insurance company is what Chambers calls “forms-intensive” — he’s full of nifty phrases. Currently, an insurance agent has to preload customer records or retrieve them from a server, which takes time. The agent has to go through the various drop-down menus in the application until he or she can retrieve the specific customer record.

If the agent could use a voice interface to access the right form, not only would it make the experience faster and less painful, but it would make it easier for the agent to fill out forms with customers in real time, which, studies show, can be a bonding experience.

Finally, two major U.S. airlines are working with SpeechWorks to design a multimodal app for deployment in a popular handheld to access flight information. Ask for flights leaving SFO for JFK, for example, and you get the list.

For the challenge of deploying multimodal, I spoke with Jeff Kunins, director of product management at TellMe Networks in Mountain View, Calif.

The hard part in deploying multimodal applications is the coordination, Kunins said. The multimodal OS or application has to know what the UI design is supposed to be doing and has to keep the back end aware of what is going on so each request executes the way it is supposed to execute.

Multimodal is also a bear to program properly, adds Nigel Beck, director of the service provider segment at IBM’s pervasive computing division in Boca Raton, Fla.