Buildkite

Buildkite#

Buildkite splits the model: SaaS control plane, operator-owned runners. The web app coordinates pipelines; the actual builds run on machines the operator owns. Strong fit for engineering teams that want SaaS ergonomics without the SaaS data-egress and isolation problems.

Architecture#

  • Pipeline, the definition of a workflow. Lives in the repo or in the Buildkite UI.

  • Build, an execution of a pipeline triggered by an event or manual run.

  • Step, a unit inside a build (command, wait, trigger another pipeline, group).

  • Agent, the operator-managed worker that picks up jobs. Runs anywhere: VM, Kubernetes, on-prem, EC2 fleet, mac mini.

Pipeline shape#

Two layers: a static YAML pipeline buildkite agent pipeline upload into the build at runtime, allowing dynamic generation.

# .buildkite/pipeline.yml
steps:
  - label: ":eslint: lint"
    command: "pnpm lint"
    agents: { queue: "linux-small" }

  - label: ":vitest: test"
    command: "pnpm test"
    parallelism: 8
    agents: { queue: "linux-medium" }
    retry:
      automatic:
        - exit_status: 137         # OOM-killed retry once
          limit: 1

  - wait

  - label: ":docker: build"
    branches: "main"
    plugins:
      - docker-compose#v5.7.0:
          config: docker-compose.ci.yml
          run: build
    command: "scripts/build-and-push.sh"

Dynamic pipelines#

A build can buildkite-agent pipeline upload a generated YAML chunk during the build itself. The shape is “first step writes the rest of the pipeline based on the diff.” Useful for monorepos where the steps depend on which paths changed.

#!/usr/bin/env bash
set -euo pipefail
changed=$(git diff --name-only HEAD^ HEAD)
if echo "$changed" | grep -q '^services/api/'; then
  buildkite-agent pipeline upload .buildkite/api.yml
fi
if echo "$changed" | grep -q '^services/worker/'; then
  buildkite-agent pipeline upload .buildkite/worker.yml
fi

Agents#

Pattern

Detail

Static fleet

A set of always-on agents in queues by capability (linux-small, gpu, mac).

Auto-scaling

The Buildkite “Agent Stack for AWS / Kubernetes / Elastic CI” autoscales agents based on queued jobs.

On-prem

Run the agent binary on operator-owned hardware; the only data sent off-site is metadata, not source.

Triggers and concurrency#

  • Pipeline triggers: push, PR, schedule, API, manual.

  • concurrency and concurrency_group on a step limit parallel runs of e.g. a single deployment job across builds.

  • Wait steps fan in; group steps fan out.

Secrets and auth#

  • Agent hooks (pre-command, environment) inject secrets from the operator’s secret store (Vault, AWS Secrets Manager) just-in-time.

  • OIDC issuance for short-lived cloud credentials.

  • Build secrets are intentionally not stored in Buildkite’s control plane; the agent fetches them locally.

When to pick Buildkite#

  • Code on any VCS; Buildkite is VCS-agnostic.

  • Source code must not leave operator-owned infrastructure; agents stay local.

  • Heavy parallel test or build workloads where the autoscaling agent fleet outperforms SaaS-runner alternatives.

When to pick something else#

  • Small team / small repo; the operator gets less for the Buildkite seat price than the GitHub Actions ecosystem.

  • Air-gapped without internet; the control plane is SaaS.

References#

Contents