The Heartbleed recovery starts with you and me

Yes, we get it. OpenSSL really screwed up — for a long time. And nobody who knew about it said anything. Meanwhile, millions of “secured” sites were helpfully vomiting up 64KB chunks of process memory just for the asking.

Yeah, it sucks.

[ Also on InfoWorld: The rise and fall of Heartbleed hysteria | How to rethink your security strategy for today’s world. Bonus: Available in PDF and e-book versions. | Stay up to date on the latest security developments with InfoWorld’s Security Central newsletter. ]

But as I wrote last week, if you keep your production systems a release or two behind current, you were probably fine. If you were running OpenSSL 1.0.1 and had a large vulnerable footprint, you hopefully had orchestration tools in place and were able to patch those systems quickly. Of course, that doesn’t help with the rekeying and cert regeneration that everyone is probably still sorting through, but at least you could stop the bleeding.

Now comes the backdraft. After everyone was able to put out their fires and get a handle on Heartbleed, we get the inevitable blame game, holier-than-thou proclamations, threats, and forking that this kind of event engenders. For instance, OpenBSD is overhauling OpenSSL. It’s likely that this will turn into an OpenBSD-centric fork of OpenSSL rather than being accepted back into the OpenSSL base — but who knows at this point?

Then there was Akamai’s claim to be invulnerable because it used a custom memory allocator it had written into OpenSSL, but not contributed back to the project. Of course, that software was later proven to be vulnerable and Akamai’s method still unsafe.

There have been lots of accusations and lots of people discussing the OpenSSL codebase as if it was a festering cesspool of awful code. There have been loud but baseless plans to reinvent open source SSL altogether. I alluded to the core of the matter at the end of my column last week: OpenSSL has been neglected for a long time. It has been taken for granted by developers and companies for more than a decade, and in the meantime, it has been implemented in two-thirds of all Web servers.

Let’s put that into perspective. Netcraft says it received responses from 861,379,152 websites — that’s a fuzzy number, but good enough. That means roughly 573,678,515 websites are running OpenSSL code. In fact, those websites rely solely on OpenSSL for any and all encryption protection. That’s a massive number of users for any software, be it commercial product or open source project — yet there’s only one full-time developer of OpenSSL.

OpenSSL has 11 team members scattered across several countries, most of whom are volunteers — the sort of team that we ordinarily see on an open source project with a far smaller footprint. Clearly, OpenSSL has been flying well under the radar, a nearly invisible but massively important cog in the economy of nearly every nation on Earth — the foundation of secure communications across the globe.

I don’t think it’s fair or righteous to be all kinds of upset at OpenSSL or the development team. With closed source code we don’t know what’s going on under the hood until a zero-day exploit suddenly comes to light. Then we’re completely beholden to the software vendor to issue a fix. That’s a company we pay directly for the privilege of getting those updates and for the software to begin with.

To those who will try to claim that Heartbleed exposes a fundamental flaw in open source software practices, actually the opposite is true. The fact that once this vulnerability was found, it was fixed almost immediately, without the need to wait for any kind of official patch from the OpenSSL team, is a testament to how well open source works.

That said, if there’s any blame to pass around for Heartbleed, it should fall on the entire software development community and software vendors themselves. Many commercial products leverage OpenSSL, but have never contributed a dime or a line of code. Nearly all Unix-like operating systems use OpenSSL, yet contribute little if nothing back.

All software gets QA’d, whether it’s by paid QA engineers at the company developing the software or by developer peers and users of open source software. Bugs like Heartbleed can and should be seen and fixed very quickly when they occur, but we have no right to expect that if only a tiny group of people are reading the code.

As I said last week, OpenSSL has been taken for granted for too long. Maybe it needs a complete rewrite, maybe it doesn’t. Maybe a few forks will ultimately do it good, or maybe the fracture will be too great to overcome. All I know is that we, collectively, are to blame for Heartbleed, if for no other reason than our ignorance.

This story, “The Heartbleed recovery starts with you and me,” was originally published at InfoWorld.com. Read more of Paul Venezia’s The Deep End blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.