A brief intro to the Pants build system

The Pants build system is a polyglot tool with a slew of nice features including full parallelism and remote worker support. Pants has been around for several years, but since version 2.0 runs on a completely rebuilt Rust engine that’s geared for speed. 

Pants is on version 2.9 as of this writing, and now includes alpha support for Java and Scala as well as support for Python and Go. Keep reading to get a look at this promising open source build tool.

Pants features

There are a few interesting features that make Pants a build system worth considering as an alternative to traditional choices. Here they are with a description.

• Automated dependency inference. Take a deep breath and let out a sigh of relief: Pants can analyse your import statements and figure out what to download without a ton of metadata.
• Fine-grained invalidation. Pants works on the smallest unit possible, so failure invalidates the minimum of new work.
• Smart results caching. Pants is designed to cache results precisely based on inputs, so the same executions over time will not require redoing the same work.
• Concurrent execution. Pants is written in Rust and architected to make full use of multi-core machines, thereby enabling parallelism in your builds.
• Remote workers. Pants can be set up on remote build machines. Combined with caching and concurrency, Pants allows teams to reuse one another’s build results and take advantage of high performance execution.
• Polyglot. Multiple languages and platforms are supported via a unified interface.
• Plug-in API. An advanced plug-in system allows plug-ins to use all these same facilities.

Dependency management in Pants

Traditional build tools like Maven or Bazel require you to maintain a set of dependencies, listed in an external file. Pants proposes to change this by analyzing your source files and inferring them. 

As you can imagine, pulling this off across wildly different platforms (Python vs. Java, for example) is no mean task. See this description of the effort in the JVM world.

In some instances, you’ll still need to supply hints to the system as to where to find external dependencies for Java. As the team notes in adding alpha support:

[…] third-party dependencies are often in JAR files that Pants won’t even download until it’s sure that it needs them. Sometimes the name of the artifact’s group can give us a clue — an artifact in the com.google.truth group may well contain com.google.truth class files — but the rest of the time, we need more data. You can supply a list of JVM packages provided by a Maven artifact when you define your dependencies, and for other artifacts, we make use of an internal database of packages to unversioned Maven coordinates.

So you’ll still have some degree of manually tracking dependencies, but less so than usual, and once the system understands how to resolve a given dependency to its remote location, Pants will use that info across all your projects within the repo. You won’t have to repeat that information as you do with Maven.

In the case of Python, you’ll similarly create a central file to map the “universe” of dependencies used by the repo, but Pants is intelligent about what dependencies are actually in use. (This in contrast to the blanket approach used by other tools.)

As the Pants docs state, “Pants understands exactly which dependencies every file in your project needs, and efficiently uses just that subset of dependencies needed for the task.” 

In Pants, you define requirements mappings in a file that, for Python, looks like Listing 1.

Listing 1. Pants python_requirements file

python_requirement(
    name="Django",
    requirements=["Django==3.2.1"],
)

There is quite a range of options for fine-tuning the dependency management engine. You’ll find the details here.

Pants goals

Similar to other command line tools, Pants uses the concept of goals as the main commands when executing. For example, you can run the simple help goal by executing ./pants help.

Goals can have sub-goals (./pants help goals), can be configured with different options, and also can be chained together, as with ./pants fmt lint ::, which runs the format goal, followed by the lint goal, configured with ::, which means all files recursively. (You can find more info on file and directory syntax for the Pants command line here.)

Pants also supports the ability to run in a “loop” mode, whereby the engine will watch for changes and rerun the commands in sequence. For example:

./pants --loop check test project/app_test.py

Notice the --loop flag.

Pants BUILD files

The metadata for your Pants builds will be defined inside BUILD files. One of the main things these files define is targets, which are named chunks of information to be applied to a goal type. Each target lives in a directory, and the path plus name provide a unique ID to address that target in configuring other targets or in parameterizing goals on the command line.

Pants backends

In pants, the various capabilities that drive goals are known as backends. You enable backends on your project in the global configuration file pants.toml. You can see how to enable a few Python-related capabilities in Listing 2.

Listing 2. Enabling Pants backends

backend_packages = [
  "pants.backend.shell",
  "pants.backend.python",
  "pants.backend.python.lint.black",
]

You can see the backend capabilities are hierarchical namespaces. The full list of available backends is seen here.

Pants source roots

Another important aspect you’ll have to configure is the source roots in your projects.  This is also done in the global pants.toml file. You can see a simple example of identifying Python and Java roots in Listing 3.

Listing 3. Source roots for Python

root_patterns = [
  '/src/python',
  '/test/python',
  ‘/src/java’
]

Project introspection in Pants

Pants includes some pretty powerful introspection capabilities to get information about your projects. 

For example, you can find all of your projects’ targets by running the list goal. You can identify all of the dependencies used by a target with the dependencies goal, or find what files a target uses with filedeps.

Of course, the count-loc goal is always handy for counting the total lines of code and estimating complexity. Goals like these also support fine-tuning the files and directories to be operated against by passing in file and dir arguments.

Pants and continuous integration

The Pants docs include quite a bit of information on how to best use the system with CI (continuous integration). You can leverage a lot of the benefits inherent in the Pants system (parallelism, speed, and caching) for your CI setup, so it’s well worth the effort to consider these options.

Pants and remote work

There are two aspects to remote work in Pants:

1. Remote caching. Pants stores the results of local process execution in a remote cache and also consumes results from that remote cache.
2. Remote execution. Pants offloads execution of processes to a remote server (and consumes cached results from that remote server).

Remote caching and remote execution are handled by setting up a remote server that exposes a REAPI compatible API, like BuildBarn.

The local Pants clients on your dev machines then interact with the remote server to take advantage of these features. For larger teams, these are potentially huge wins for dev productivity.

Pants language support

The ultimate goal of Pants is to provide a one-stop build experience for all of your programming languages. This would make it the ultimate monorepo build system, where you could just throw anything at the tool and it would understand it.

As of this writing, Pants has first level support for the following:

• Python – Since version 2.0
• Protobuf and gRPC – Since version 2.2
• Shell – Since version 2.5
• Go – Since version 2.8
• Java – Alpha support beginning in version 2.9
• Scala – Alpha support beginning in version 2.9
• JavaScript – On the roadmap

Pants is an ambitious project. If it continues to add languages, it could offer an incredibly new and simpler way to build projects, especially for large projects with several languages in use.