The technology for maintaining consensus and distributing data across a cluster now powers more than just container-based projects Credit: Thinkstock CoreOS’s ETCD, a key component of container orchestration technologies and many other kinds of projects as well, has now debuted a more efficient and scalable 3.0.0 release. ETCD is a distributed key-value store that’s used by CoreOS to maintain state and provide consensus between the many nodes of a cluster. Aside from being used by CoreOS’s namesake project, it’s also used in projects like Torus, CoreOS’s storage provisioning system for containerized apps. Kubernetes, the Google-created container orchestration system that also powers other products, also uses ETCD as a key component. The blog post announcing ETCD 3.0.0 describes how the single biggest change to ETCD was a new API and data model, resulting in better performance and a smaller memory footprint. Instead of using JSON as the messaging protocol, ETCD 3 uses the Google-developed gRPC, which uses HTTP/2 to speed connectivity between clients and is generally faster to process than JSON. To further speed things up, CoreOS combined gRPC with some new mechanisms for handling the lifetimes of keys held in its store, and for waiting on changes to keys. Originally, ETCD’s ways of handling these things caused a great deal of network chatter and other unwanted side effects With these last features in particular, CoreOS is trying to draw contrasts not only with previous versions of ETCD, but with other competing technologies for performing distributed coordination across a cluster. Two mentioned in particular are Apache’s ZooKeeper (used by Netflix, among others), and Consul from Vagrant creators HashiCorp. CoreOS originally made a name for itself by way of its super-slender Linux distribution, designed to do little more than run containers at scale, and its RKT container runtime, intended to bolster container security — something it saw as being an underaddressed issue. The company was also a key player in helping formalize standards around container image formats. With projects like ETCD, though, CoreOS is building infrastructure that’s meant to have broader implications than in the container world alone. ETCD can in theory be used to maintain consensus between nodes for any kind of workloads, and several third-party projects that aren’t directly related to container management have picked ETCD as a core component — such as Huawei’s Cloud Foundry-based PaaS, or NTT’s Elastic Services Infrastructure. Open SourceTechnology IndustryCareersSoftware Development