Paul Krill
Editor at Large

Facebook open-sources its Chef cookbooks

news analysis
Oct 8, 20142 mins

Facebook offers two homemade devops tools for systems management on GitHub and says more are on the way

Having developed its own devops tools to deal with the management of hundreds of thousands of servers, Facebook is now making these tools available to others via open source.

The company already has been offering two of its Chef “cookbooks,” which work with the Chef configuration management platform, and plans to make more available over time, said Phil Dibowitz, Facebook production engineer, in an interview on Tuesday.

“We use [Chef] for managing the OS and configuration for everything from middleware all the way down the system for our entire fleet,” Dibowitz said. This includes configuration of the operating system and middleware layers and package installation.

Accessible on GitHub, the two cookbooks released are for running scheduled jobs and for system controls, including setting up defaults, adding new system controls, and changing existing ones. Chef shops running a lot of servers can scale their infrastructure by using Facebook’s tools. “The upside is that it sets you up to scale incredibly well, but the downside is that it’s a different model from the way a lot of people use Chef,” Dibowitz said.

Facebook also has released its Taste Tester testing framework and re-released its Grocery Delivery tool, which watches a source code repo, such as Git, and keeps the local Chef server in sync. Other cookbooks are expected to be released over time for such functions as controlling which DNS server is used. “Certainly, it is our goal to release as much as possible,” said Dibowitz.

Facebook’s Chef platform serves as a devops mechanism, with devops geared to shared responsibility between systems personnel and code writers. Engineers were required to write cookbooks describing their systems, which provided for shared knowledge, Dibowitz said.

With its switch to Chef, Facebook was looking for simplicity in making changes to systems. “What we really wanted to do three years ago was build a system that didn’t require one central team to understand every possible combination of configurations that could exist throughout the entirety of Facebook because that doesn’t scale very well when it comes to people,” Dibowitz said.

Instead, Facebook wanted a system where a small, central team could build a set of APIs that allowed engineers who owned any system to describe what they wanted, such as a DBA running MySQL server who wants more shared memory, he explained. A one-line variable should be all that was necessary to make that happen.

Paul Krill

Paul Krill is editor at large at InfoWorld. Paul has been covering computer technology as a news and feature reporter for more than 35 years, including 30 years at InfoWorld. He has specialized in coverage of software development tools and technologies since the 1990s, and he continues to lead InfoWorld’s news coverage of software development platforms including Java and .NET and programming languages including JavaScript, TypeScript, PHP, Python, Ruby, Rust, and Go. Long trusted as a reporter who prioritizes accuracy, integrity, and the best interests of readers, Paul is sought out by technology companies and industry organizations who want to reach InfoWorld’s audience of software developers and other information technology professionals. Paul has won a “Best Technology News Coverage” award from IDG.

More from this author