Run builds on a dedicated node

A build runs the commands in a repository's Dockerfile. The build job is confined, but it still runs on a node's kernel. On a cluster with more than one node, reserve a node for builds so that code never shares a kernel with your workloads or with the agent.

Before you beginCopy Anchor Link

• The cluster has at least two nodes. On a single-node cluster, builds and services share the node, and nothing here applies.
• The node you reserve has enough free disk for builds. A build can use about 25 GiB of scratch space at the default limits. See Change build limits.
• You have shell access as root to a server node of the cluster.

How placement worksCopy Anchor Link

Without a dedicated node, a build runs on any node. The agent and its access broker prefer other nodes than the one running a build, but run beside it when the cluster has no other node.

When any node carries the label nebula.dev/build-node=true, every build that starts afterwards runs only on labelled nodes, and tolerates the taint nebula.dev/build-node=true:NoSchedule. A build never falls back to another node. If every labelled node is down or full, the build waits until its 30-minute limit and fails.

A build that is already running keeps its node.

StepsCopy Anchor Link

k3s kubectl label node NODE nebula.dev/build-node=true

k3s kubectl taint node NODE nebula.dev/build-node=true:NoSchedule

VerifyCopy Anchor Link

Start a build, then list the node of its pod:

k3s kubectl -n nebula-build get pods -o wide

NAME                 READY   STATUS    RESTARTS   AGE   IP           NODE
build-1f3c9a2e-x7k2p   3/4   Running   0          41s   10.42.2.17   builder-1

The NODE column shows the labelled node.

Undo itCopy Anchor Link

Remove the taint and the label. Builds then run on any node again.

k3s kubectl taint node NODE nebula.dev/build-node=true:NoSchedule-

k3s kubectl label node NODE nebula.dev/build-node-

Next stepsCopy Anchor Link