Ex-Amazonian urges Google to sample Amazon’s secret sauce

A Google+ post-gone-viral by Google engineer Steve Yegge includes both insightful and at times scathing observations about how his employer needs to learn a lesson or three from his previous employer Amazon, as well as Facebook and other leading tech companies about the importance of building a services-oriented technology platform instead of cranking out stand-alone products.

Privacy concerns aside (Yegge did opt to remove his public-facing post at his own discretion), observers may be fascinated by the contents of the post (reposted here on ycombinator), in which Yegge provided some deep insights into Amazon’s successful transformation into an SOA company, despite what he described as occasionally harsh and misguided direction of micromanagerial Jeff Bezos.

Further, Yegge opined that Google needs to learn a particularly critical lesson from Amazon, as well as Facebook, Apple, and other tech giants before it’s “too late.” “Start with a platform, and then use it for everything,” Yegge urged, citing Google+ as a prime example of his employer wrongly creating a mere product instead of a Facebook-style platform upon which products can be added and connected, services-style.

Yegge’s assessment of Amazon and its path to SOA is particularly interesting, in part because it comes through the lens of a computer engineer. In a nutshell, he faults Bezos for his at times shortsighted and heavy-handed approach to what ultimately was the company’s successful service-focused transformation.

Yegge summarized Bezos’s “Big Mandate” thusly:

1. All teams will henceforth expose their data and functionality through service interfaces.
2. Teams must communicate with each other through these interfaces.
3. There will be no other form of interprocess communication allowed: No direct linking, no direct reads of another team’s data store, no shared-memory model, no back doors whatsoever. The only communication allowed is via service interface calls over the network.
4. It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter. Bezos doesn’t care.
5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
6. Anyone who doesn’t do this will be fired.

What followed, per Yegge: Employees carried out Bezos’s bidding as best they could, grappling with the reality that SOA was still in its infancy and documentation and best practices were few and far in between. Thus, the IT teams ran into all sorts of headaches and challenges. Among them:

• Service calls took longer to address because it was difficult to track down the owner of a particular problematic service.

• “Every single one of your peer teams suddenly becomes a potential DOS attacker” because they had to set serious quotas and throttling for every single service.
• The challenge of viewing monitoring and QA as one in the same. Just because a service says it’s fine doesn’t mean the whole service is indeed fine; rather, it could be that just a component in a server is telling you everything is fine. “In order to tell whether the service is actually responding, you have to make individual calls. The problem continues recursively until your monitoring is doing comprehensive semantics checking of your entire range of services and data, at which point it’s indistinguishable from automated QA. So they’re a continuum,” Yegge wrote.
• They discovered the need for both a service-discovery mechanism and a service-registration mechanism, to ensure that all services among different groups can find and communicate with one another “So Amazon has a universal service registry where you can find out reflectively (programmatically) about every service, what its APIs are, and also whether it is currently up, and where,” Yegge wrote.
• Debugging problems with someone else’s code becomes far more difficult in an SOA environment, Yegge noted: “It’s basically impossible unless there is a universal standard way to run every service in a debuggable sandbox.”

Yegge does concede that Amazon’s approach has paid off, in part because Amazon succeeded in laying out a solid SOA platform on which it can now easily add new services. This, in turn, has enabled Amazon to move beyond selling goods to providing its successful cloud services, such as Amazon Elastic Compute Cloud, Amazon Elastic MapReduce, and the Amazon Relational Database Service. “There are, without question, pros and cons to the SOA approach, and some of the cons are pretty long. But overall it’s the right thing because SOA-driven design enables platforms,” he wrote. “That’s what Bezos was up to with his edict, of course…. He realized long before the vast majority of Amazonians that Amazon needs to be a platform.”

Google, Yegge argued, doesn’t “get” platforms; rather, the company has traditionally remained focused on individual products without considering how the individual items tie together. “A product is useless without a platform, or more precisely and accurately, a platform-less product will always be replaced by an equivalent platform-ized product,” he wrote.

Yegge cited Google+ as “a prime example of our complete failure to understand platforms from the very highest levels of executive leadership … down to the very lowest leaf workers.”

“The Google+ platform is a pathetic afterthought. We had no API at all at launch, and last I checked, we had one measly API call…. Google+ is a knee-jerk reaction, a study in short-term thinking, predicated on the incorrect notion that Facebook is successful because they built a great product. But that’s not why they are successful,” Yegge wrote. “Facebook is successful because they built an entire constellation of products by allowing other people to do the work. So Facebook is different for everyone. Some people spend all their time on Mafia Wars. Some spend all their time on Farmville. There are hundreds or maybe thousands of different high-quality time sinks available, so there’s something there for everyone.”

This story, “Ex-Amazonian urges Google to sample Amazon’s secret sauce,” was originally published at InfoWorld.com. Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest developments in business technology news, follow InfoWorld.com on Twitter.