Review: Chef cooks up configuration management

Among the various configuration management frameworks available today, Opscode’s Chef is a relative newcomer. But as the double-digit version number would suggest, Chef has been making strides toward adding new features and appears to be gaining some ground against its older and more mature competitors.

For any shop considering Chef for configuration management, the main question will be the level of familiarity and comfort with the Ruby programming language, which Chef relies on for driving all configuration changes.

[ Review: Puppet Enterprise 3.0 pulls more strings | Puppet or Chef: The configuration management dilemma | 12 clever cloud tools for devs and ops | Subscribe to InfoWorld’s Data Center newsletter to stay on top of the latest developments. ]

Chef is available in a variety of different flavors. Opscode offers Hosted Chef, which is a hosted service that Opscode runs in the cloud and allows you to implement Chef without running your own local Chef server. Then there’s Opscode’s Private Chef, a commercial product that installs locally and offers an extended selection of features such as high availability and an enhanced Web UI. Lastly, there’s the open source Chef, which is free but lacking in enterprise-grade features and somewhat more challenging to set up and deploy. For this review, I’ll focus on Private Chef.

Each Chef installation requires a Chef server, a Chef workstation, and nodes that connect to the Chef server for management. Regardless of whether you’re using Hosted Chef, Private Chef, or open source Chef, this baseline configuration is required. Smaller builds might get away with combining the Chef server and workstation on the same hardware, but larger deployments will need to separate those functions, and perhaps even separate the Web UI servers if operating in a high-availability or clustered Chef environment.

Installing Chef Building and installing Chef is fairly straightforward once you understand the overall layout and use of certificates, so long as you have a solid background in Unix/Linux administration and scripting. The documentation offered by Opscode is reasonably clear for the most part, but quite vague in some places, which can cause problems for the uninitiated.

The basic concept is that you install the Chef server on a supported Linux distribution, which means either RHEL/CentOS or Ubuntu Server. This is simple, requiring only the installation of a few packages. Then you run a reconfiguration using the private-chef-ctl tool, followed by a test. Assuming all goes well, you now have a functional Chef server.

Next up, the Chef workstation needs to be configured. This is accomplished by pulling down an omnibus installer script from Opscode that determines the distribution and installs the requisite workstation packages. From there, the knife utility is used to configure the workstation and user. This requires pulling several configuration files and certificates from the Chef server to the workstation and placing them in specific locations. Once this is done and the installation questions (such as the URL for the Chef server) have been answered, the workstation should now be able to communicate with and control the Chef server.

At this point, we have a fully functional Chef installation, and we can start adding nodes. We can also log into the Web UI on the Chef server using the username and password created during the installation procedure.

Adding nodes in Chef is simple. On the workstation, the knife bootstrap command is used to log into the target system via SSH and deploy the Chef client. It will also configure the client and server. Thus, adding a node to Chef is a scriptable one-liner, driven from the workstation. Chef does support Windows, but the Windows installer needs to be run on the client, as it obviously cannot be run through SSH by default. There are some community Cookbooks for Windows management and a Ruby gem for bootstrapping Windows clients, but they are not part of the default installation.

Once several nodes have been added, we can begin to install and modify configuration elements for those nodes. Chef makes use of Git to handle configurations. On the workstation, you use Git to clone the chef-repo repository, and from there, you download and create configuration elements for the deployment. All configuration is handled within the purview of Git repositories.

Using the Web UI The Web UI of Private Chef goes a little beyond that for the open source server, but it’s still fairly rudimentary. Notably, Private Chef allows the creation and management of user groups through the Web UI, and it supports centralized LDAP and Active Directory authentication, whereas the open source version does not. Private Chef also includes features such as push commands and centralized reporting, though these are still in beta, and the documentation and availability of these options are slim. In fact, push commands could not be evaluated for this test because they are “not ready for public consumption,” according to Opscode.

The Web UI in Private Chef provides views of nodes, node activity, and roles, along with the ability to create environments and groups and assign nodes to those objects. You can also view the list of installed Cookbooks and assign them to nodes or roles. Otherwise, all configuration is handled via the command line on the Chef workstation.

Roles are groups of nodes that should have the same Cookbooks assigned to them. We might have a Web server role and a Database server role, for example, or any other logical group. You can then assign nodes to those roles, and Cookbook recipes as well, which will cause those recipes to run on nodes assigned to those roles.

Part of the chef-client run sends inventory data to the server. Through the Web UI you can view the inventory of a node, such as the CPU count and type, RAM size, file system type, kernel version, and so forth. You can also search for these elements through all nodes.

The search tool also allows you to search within environments, roles, clients, and “data bags” (JSON objects Chef uses to store data used by Chef recipes).

Using Chef Cookbooks Cookbooks are Chef’s configuration modules. They are used to craft configuration management elements to be deployed to nodes connected to the Chef server. Cookbooks are downloaded through Git on the workstation and wrapped by the knife command, which is the general Chef management tool. All Cookbooks are built in Ruby, as Chef uses Ruby for all Cookbook functions. This means that administrators of a Chef environment must be well versed in Ruby to build their own Cookbooks, though modifying existing Cookbooks will be somewhat less challenging. This dependence on Ruby is both a positive and negative aspect of Chef, as Ruby is not necessarily a common language for sys admins, and it can present quite a learning curve for nonprogrammers.

As an example, we may want to install a Cookbook to ensure that the NTP service is installed and configured for our environment on all of our nodes. To do this, we would use the knife command to download the NTP Cookbook:

knife cookbook site install ntp

This will place the NTP Cookbook in our chef-repo/cookbooks directory and set it up. From there, we need to dig into the Cookbook to make sure our preferred configuration settings are present.

Cookbooks have a specific file layout consisting of several directories such as “files,” to store whole files that may need to be copied to the node; “templates,” to house templates for the creation of new files with specific parameters; “attributes,” to contain files that are parameter definitions; and “recipes,” to store the Ruby scripts that are run to combine all of the above into a functioning Cookbook.

To continue with our example, the NTP Cookbook includes an attributes directory that contains a file named default.rb. This is where we would make changes to the default NTP servers. In this case, that’s as simple as changing the default['ntp']['servers'] variable by adding our NTP servers:

default['ntp']['servers']   = %w{ 192.168.32.10, 192.168.16.16 }

We might change other parameters in this file to modify the various directory paths or the user and group the service runs under, for example.

In the recipes directory, we find a number of Ruby scripts that perform the actual configuration functions. These include scripts that ensure the NTP package for the target distribution is present on the system, scripts that use the template files in the template directory to build new configuration files, and scripts that ensure the service is stopped or started, depending on the configuration.

In this simple example, the Ruby scripts will install the NTP package if necessary, construct a new ntp.conf file by using the template, substitute the configuration parameters specified in the default.rb file under attributes, then start the ntpd service and verify it’s running.

Once we’ve configured this Cookbook to our requirements, we need to upload it to the Chef server for deployment. This is done via the knife command:

knife cookbook upload ntp

This will upload the Cookbook to the server. Then through the command-line or the Web UI, we can assign that Cookbook to our nodes.

Applying node configuration Because there’s no functional push command available in Chef, we need to make sure that each node runs the chef-client executable on a scheduled basis. Although most configuration management tools set this up automatically, Chef does not. Thus, we need to add a cronjob on each node to have that node check in with the server periodically. If this isn’t done, then even after a node has been bootstrapped, it will not check in with the server again. We can automate check-ins by using the chef-client Cookbook that adds a cronjob, but it’s not done by default.

Assuming we have our nodes set up to run chef-client every half an hour, once our NTP Cookbook has been applied to the node, it will run, and our NTP service will soon be configured.

Opscode provides a large collection of ready-to-mix recipes in its GitHub repository, and chances are you’ll find existing Cookbooks to match a significant number of common configuration tasks, but you’ll need to bust out your Ruby skills to make your own from scratch.

Alongside Cookbooks, Chef has the concept of a data bag — essentially a JSON object containing any number of global variables. These variables can be accessed by Cookbooks to use when running recipes. You can create data bags with the knife command or manually or through the Web UI, though you will likely find that creating the JSON objects manually, then using knife data bag from file mybag filename.json is easiest when working with large numbers of variables.

Once you have data bags in place, your recipes can use them to populate variables during runtime — for instance, ensuring that certain user accounts are created on a new system with specific passwords and so forth. Data bags can be encrypted using a shared secret for protection of sensitive information.

For many system administrators who are not also programmers or who lack significant programming experience, Chef will seem more obtuse and more difficult to learn than other solutions. It also currently lacks functional push commands, as well as reporting tools offered by other solutions. That said, Chef may be a better fit for development-minded admins and development shops in general, as the full Ruby scripting required to create Cookbooks will be more familiar.

This article, “Review: Chef cooks up configuration management,” was originally published at InfoWorld.com. Follow the latest developments in application development, cloud computing, and open source at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.