Once a software project becomes large enough, you often see that the
developer experience becomes painful. Builds tend to take ages. In the
worst case, it can even become impossible to do a full 'build and
test' on your personal machine. A full build may depend on different
hardware platforms and operating systems. You often see that people
then resort to building just parts of the project on their own system,
and hope that CI (Continuous Integration) catches any remaining
problems. This is of course undesirable.
Blaze is a build system that was developed inside of Google to solve
this problem. What makes Blaze an interesting tool is that it only
orchestrates builds. Instead of running build actions (e.g.,
compilers) on your own system, it sends requests to a build cluster.
Builds become a lot faster as a result of that, both because clusters
are faster than a single workstation and because they can maintain a
centralized cache.
In 2015, Google released an Open Source version of Blaze called
Bazel.
One of the highlights of the project has been a community-driven
effort to design an open build cluster protocol, named 'Remote
Execution v2' (REv2 for short). It looks like this protocol is
becoming the de facto standard for doing remote builds. There are now
various clients (Bazel, Pants, Goma, Recc, Buildstream) and servers
(Buildbarn, Buildfarm, Buildgrid and Google Cloud RBE) that implement
it.
In this talk I'm going to discuss how REv2 works. By the end of this
talk you will have a good understanding of the data model that Bazel
and Buildbarn use, and how they exchange inputs (source code) and
outputs (object files, executables).
Ed's slides are now available
By: Ed Schouten