Google’s microservices protocol joins Kubernetes in cloud foundation

Google’s gRPC protocol was originally developed to speed up data transfer between microservices, proving faster and more efficient than passing around data encoded in JSON.

Yesterday the Cloud Native Computing Foundation (CNCF), which oversees the development of Kubernetes, announced it would also become the home for gRPC’s development.

Under the CNCF, gRPC will be broadly supported to provide a consistent, high-performance mechanism for wiring together the polyphony of technologies used in microservices—the “multiple languages, frameworks, technologies, as well as multiple first- and third-party services,” as the CNCF said.

gRPC is meant to serve as more than a channel for data. It uses protocol buffers to serialize and structure data at high speed, without being tied to any particular language. Data’s transmitted via HTTP/2 connections, so everything is encrypted in flight and kept compact thanks to the binary compression used by HTTP/2.

The CNCF, spearheaded by former Googler Craig McLuckie and currently under the wings of the Linux Foundation, was devised to provide a home for standards around next-generation cloud computing. At first this mainly consisted of container technologies—such as Kubernetes—but it’s since become home to a gallery of projects that deal with the different layers of this new ecosystem. Logging (Fluentd), service discovery and visibility (Linker-d), application tracing (OpenTracing), and metrics (Prometheus) are all addressed by projects in the CNCF’s portfolio.

gRPC is a good fit for this portfolio; it doesn’t directly overlap any of the existing CNCF projects or render any of them redundant. If anything, gRPC could serve as a substrate between many of the projects and provide a common method for developers to stitch them together.

A small but influential circle of users has already started using gRPC, although some are already close to the CNCF. For instance, gRPC is in use as the messaging protocol between nodes in Docker 1.12. And CoreOS uses gRPC as the messaging system in its etcd key-value store, to synchronize states across clusters in its container-based Linux distribution. The CNCF notes that other users of gRPC include Lyft, Netflix, Square, Cisco, Juniper, Arista, and Cockroach Labs (creators of CockroachDB).

gRPC might also be put to use as a standard ingredient for other projects. Recently, the RethinkDB project was purchased by CNCF and donated to the Linux Foundation, after its commercial owners failed to monetize it, most likely because its scale-out architecture makes it a good fit for cloud projects. RethinkDB currently uses a piece of gRPC—Google’s “protobuf” specification—for its wire protocol. If RethinkDB was retooled to use gRPC for more functions, the existing developer base would have related experience with protocol buffers, and the CNCF will be better able to support such a move.

[An earlier version of this article stated that RethinkDB was acquired by the CNCF, but implied that the CNCF retained control of the project, and did not note that its code was donated to the Linux Foundation.]