Toyota’s unhappy introduction to the software business

Buggy software often crashes, but some crashes are more serious than others. When Apple co-founder Steve “Woz” Wozniak noticed a reproducible problem with the cruise control on his Toyota Prius that caused the car to accelerate in unexpected ways, he was understandably concerned. His Prius, it seemed, was falling prey to a software bug — a potentially dangerous one.

Woz had reason to be alarmed. Even now, Toyota is embroiled in the greatest crisis in its history, having recalled some 8.5 million vehicles on five continents to repair defects linked to cases where cars have accelerated out of control. Unexpected acceleration is now believed to have played a role in at least 815 collisions and as many as 34 deaths.

[ Keep up on the latest developments and insights on software development in InfoWorld’s Developer World newsletter. ]

Word of Wozniak’s complaint spread far and wide, eventually reaching the ears of Toyota president Jim Lentz, who offered the legendary hacker his private phone number. But while Woz’s story has a happy ending, the full chronicle of how Toyota has responded to the acceleration issue is a troubling one. Unfortunately, it represents a pattern that will seem all too familiar to software users everywhere.

Toyota’s first reaction to reports of acceleration problems on its cars was to blame user error. That is, it ignored the issue. Only after the Los Angeles Times reported that the number of fatalities linked to acceleration problems in Toyota vehicles since 2001 was greater than the number of fatalities for any other automaker did Toyota acknowledge that it was investigating the matter.

Under federal law, automakers must notify the National Highway Traffic Safety Administration within five days of determining that a safety defect exists and promptly conduct a recall. Congress has since demanded Toyota’s records to determine how long the automaker knew about the acceleration problems before acting.

Toyota’s next step was to suggest a hardware fix. The automaker’s initial suggestion — that gas pedals on its vehicles were sticking to ill-fitting floor mats — convinced almost nobody. As pressure mounted, Toyota then assured customers that it had a permanent fix to the problem, one that involved inserting a metal reinforcing rod into the gas pedals of affected car models. The recalls commenced.

Meanwhile, complaints persisted that problems on some vehicles might involve more than simple mechanical failure. With prodding, a Toyota internal investigation revealed that there was indeed a problem with the electronic braking systems on some models. The company has since issued a firmware update to correct the braking software on the 2010 Prius.

By now, word of Toyota’s problems had hit the mainstream press, and the issue escalated from a routine defective-product situation to a full-scale PR boondoggle. As public outcry grew, Toyota hired the independent engineering firm Expedient to conduct a thorough investigation of the electronic throttle-control software on multiple auto models. Last week, Toyota executives testified before Congress that it found no faults in the software. But Toyota, apparently no longer convinced, has said that it will order a second round of tests.

When PR is a higher priority than QA So should Steve Wozniak feel vindicated? Not really. Ironically, it turns out Woz’s problem wasn’t a software issue at all, but a simple case of user error. According to John Voelcker of GreenCarReports.com, Wozniak didn’t understand that the cruise control system on his Prius works differently than the systems on many other cars. After speaking with Toyota’s Lentz, the two were able to settle the matter.

Yet that’s the whole trouble. Toyota’s handling of Wozniak’s complaint only underscores the automaker’s mismanagement of the acceleration issue from the very beginning. There are software issues in Toyota cars, but Wozniak’s isn’t one of them. Wozniak, by virtue of his fame, had access to the press. Because of that, Toyota was forced to listen — even though the complaint was an insignificant one. But if Toyota had weighed complaints on their merits from the start, it might have uncovered some of the defects that are now triggering recalls of vehicles from as far back as 2001. Instead, it waited until a looming public relations disaster forced its hand.

Auto industry analysts are rightly mystified by Toyota’s behavior. This is not a sloppy company. Until now, the Toyota brand has been virtually synonymous with quality, and it has handled past defect issues promptly and effectively.

So why is Toyota only now investigating the possibility that software defects in its cars might be partly to blame for crashes? Honest programmers will admit that it’s nearly impossible to guarantee that any software will be completely free of bugs. Should automotive software be any different? As the issue unfolds and the number of recalled Toyota vehicles mounts, industry watchdogs are now investigating whether electronic control systems deserve closer regulatory scrutiny.

The bigger issue, however, is that Toyota isn’t alone in how it handles customer complaints. The cycle of ignoring the issue, blaming the user, suggesting hardware fixes, and then only grudgingly admitting a problem is all too common in the software industry. Microsoft is a repeat offender — on many occasions it has failed to act on critical bugs for months. Adobe too is guilty. These are just the biggest examples; they are not alone.

Software vendors can do better. The rise of the Web has ushered in a golden age of communications. Yet while almost daily we hear how companies plan to use social media for marketing and PR purposes, customer communication in the other direction is still given short shrift. Too often, bug reports and other customer complaints are treated like hostile attacks, when in fact they represent valuable business intelligence. The key is to listen.

This article, “Toyota’s unhappy introduction to the software business,” originally appeared at InfoWorld.com. Read more of Neil McAllister’s Fatal Exception blog and follow the latest news in software development at InfoWorld.com.