Battling the bugs

SHOULD WE expect software to be bug-free?

Most observers of the software industry, including me, have always said the answer to this question is no. Software development is hard stuff, after all, and it’s still more art than science. So software has always seemed like the one business where customers have to accept a few defects.

Lately, however, I’m hearing from readers who do not agree that software should be treated differently. “I no longer feel that bugs are a necessary evil in software,” wrote one reader recently. “There are now procedures and tools for software development that can virtually eliminate bugs. One only has to look at the great variation in how trouble-free software from different suppliers is … I think the best practice for handling bugs could be taken from the auto industry. Nobody accepts new automobiles that can’t safely perform their basic functions — start, stop, run, protect the passengers, and meet certain minimum standards for performance. Software ought to do the same.”

This growing intolerance for bugs seems to be based primarily on the belief that the software industry has reached a level of maturity, if not downright monotony, where the cutting edge has become rather dull. Most major releases contain little new functionality that anyone really wants other than staying current with new hardware and/or OS platforms. So the least the software publisher could do, readers argue, is to fix its old bugs when they come out with a new version.

“Some bug fixes are like the tomorrow that never comes,” wrote a software procurement officer. “You’re told it will be fixed in the next release, but [the same bug] stays there generation after generation … Considering that usually all they’re adding are a few irrelevant bells and whistles, why not at least clean up the old code?”

If customers stop accepting buggy software, many readers believe, the publishers will have to clean up their act. “Why do we put up with vendors that ask us to pay money for buggy software in the first place?” another reader asks. “That is my major gripe in the entire computer arena. There is no other industry in the world that would survive if it produced the kind of shoddy merchandise that we routinely purchase and then anguish over when it doesn’t perform as advertised. Until consumers find a way to put some teeth into their complaints, we will be content to vent our frustrations by sending messages to you — unfortunately, an exercise in futility.”

One obvious factor making readers more sensitive to software defects is the growing demand for secure systems. “It seems like every day they find another major security hole in Windows,” wrote one exasperated IT manager. “What if the bad guys find one of these holes first? How can you protect yourself against something like that?”

But given that customers want to put their foot down on bugs more than ever, that still leaves the question of whether software companies are capable of delivering better software. Virtually bug-free software is feasible in those high-risk applications where the need for safety mandates that the time and resources be spent to produce it. But if all publishers were to aim that high, it’s doubtful that there would be enough qualified software engineers, software testers, technical editors, etc. to go around. And even if there were, what about the cost? The software industry has long argued that bugs are inevitable because of the 80-20 rule. You can find and fix 80 percent of the bugs for 20 percent of the cost of getting them all, they say, but beyond that it becomes prohibitively expensive. Even if that’s exaggerated, there’s no question that there are limits to how thoroughly software can be tested and debugged without driving up the costs beyond anyone’s willingness to pay.

If completely bug-free software is not yet practical, does that mean customers are stuck with the low-quality products they’re usually getting for their money now? Must we just live with long lists of bugs that never get fixed?

One expert whose opinion I trust says that software companies can deliver better software if customers demand it. “I don’t believe that costs would be substantially higher for higher quality,” says Cem Kaner, professor of computer sciences at Florida Institute of Technology and noted authority on software testing. “Many of the products that add little new functionality are coping with very old code bases that are so full of twine and Band-Aids that every change causes new problems. But even with a bad code base, the company can still document its bugs carefully, which will save many customers a lot of troubleshooting expense, and will save some customers from ever experiencing the problems. At a certain point, the problems that stay in the code year after year either reflect a judgment of the company that no one cares about them or they reflect a lack of integrity. Or both.”

Kaner, whose practical experience dealing with the quality issues faced by software publishers goes back to the WordStar days at Micropro, thinks the problem is more basic than insufficient product testing. “I think testing is fundamental, and weak testing has been the cause of serious problems. But companies can spend 50 percent of their development budget on testing and still ship bad software. Product quality is driven more by the sense of integrity and commitment throughout the company than by the skill and effort of any test team. The best testing imaginable won’t help a product much if customer satisfaction is not one of the publisher’s fundamental motivators.”

So should we expect software products to be bug-free? Yes, I think we must. If we do, when a product falls short of the mark, we can also expect the publisher to take responsibility for fixing it. If we don’t, we have no one but ourselves to blame for all those bugs.