tom@techcolumnist:~$

~/2026/03/18/talos-linux-bootstrapping-a-three-node-kubernetes-cluster-from-bare-metal.md

Talos Linux: Bootstrapping a Three-Node Cluster on Bare Metal

---
author: 
date: 
updated: 
read: 7 min
in:   [engineering]
tags: [talos, kubernetes, cluster]
---

$ grep -n '^#' post.md

Bringing up a Kubernetes cluster on three bare-metal boxes used to mean an afternoon of installing a base OS, patching it, disabling swap, installing a container runtime, and then finally running kubeadm and hoping the CNI came up clean. Talos Linux collapses most of that into a single step: there's no shell to install anything into, because there isn't a shell. The OS is an immutable, API-managed image whose only job is to run Kubernetes. The first time you bootstrap a cluster this way, the absence of familiar steps is more disorienting than the process itself.

This is the sequence I use for a three-node cluster where every node is both control plane and worker, written against Talos 1.14 (the current release, whose talosctl gen config defaults to Kubernetes 1.37.0). Talos 1.14 generates a multi-document machine config, so the patches below use the new per-topic documents (LinkConfig, Layer2VIPConfig, KubeNodeConfig, UnattendedInstallConfig) rather than the older machine.network and machine.install fields. The addressing plan is a single VLAN, 10.10.20.0/24: nodes on .11, .12 and .13, a shared API address on .10, gateway on .1.

Boot into maintenance mode and look before you configure

Download an ISO from the Image Factory. If the hardware needs extra drivers or you want system extensions, build a schematic there first, because the installer image you upgrade with later has to carry the same schematic. Boot each machine from it. Until it receives a config, the node runs entirely in RAM in maintenance mode and doesn't touch its disks. Sidero Labs is explicit that the maintenance API is unauthenticated by design: anyone who can reach port 50000 can push a config with --insecure, so keep that VLAN closed while nodes sit in this state.

Before writing any patches, ask each node what hardware it actually has:

bash
talosctl get disks --insecure --nodes 10.10.20.11
talosctl get links --insecure --nodes 10.10.20.11

Note each node's system disk serial and its NIC's MAC address. I match both by stable properties instead of /dev/sda and enp1s0, because device names on bare metal change when someone adds a disk or a firmware update renumbers PCI devices. The production notes say the same thing about disks: prefer disk.serial or disk.wwid over the /dev/sd* name.

Secrets first, then patches, never a hand-edited config

The part I got wrong on my first cluster was treating controlplane.yaml as the thing to keep. Talos' own guidance has since become explicit: generate a secrets bundle once, keep small patch files in version control, and regenerate the full machine configs whenever you need them instead of committing them. The reason is drift. talosctl upgrade-k8s rewrites component image versions in the live config on each node, so a full controlplane.yaml in Git goes stale after the first Kubernetes upgrade, and re-applying it later can quietly downgrade components.

The inputs that make a config reproducible, per the reproducible machine configuration guide, are the secrets.yaml bundle, the patches, the cluster name and endpoint, the Kubernetes version, and a fixed --talos-version contract. secrets.yaml holds the cluster CAs and bootstrap tokens: it belongs in a secrets manager or an encrypted store, not in plain Git.

The repository layout is small:

text
cluster-lab/
  bootstrap-cluster.sh
  secrets.yaml            # not committed in plain text
  patches/
    common.yaml
    controlplane.yaml
    nodes/
      talos-cp1.yaml
      talos-cp2.yaml
      talos-cp3.yaml

patches/common.yaml holds what every node shares, DNS and time:

yaml
apiVersion: v1alpha1
kind: ResolverConfig
nameservers:
  - address: 10.10.20.2
  - address: 10.10.20.3
---
apiVersion: v1alpha1
kind: TimeSyncConfig
ntp:
  servers:
    - ntp1.example.internal
    - ntp2.example.internal

patches/controlplane.yaml holds the two decisions that make this a three-node, all-in-one cluster. The first document is a Layer 2 virtual IP for the Kubernetes API. The control plane nodes elect an owner through etcd and that node answers for 10.10.20.10 with gratuitous ARP, so there's no external load balancer to build. The second removes the default NoSchedule taint from control plane nodes so workloads can run on them, using the $patch: delete form from Talos' "workloads on control plane" guide:

yaml
apiVersion: v1alpha1
kind: Layer2VIPConfig
name: 10.10.20.10
link: net0
---
apiVersion: v1alpha1
kind: KubeNodeConfig
taints:
  node-role.kubernetes.io/control-plane:
    $patch: delete

If you plan to run MetalLB or another load balancer that should use these nodes as backends, also delete the node.kubernetes.io/exclude-from-external-load-balancers label the same way; Talos adds it to control plane nodes by default.

Each node then gets its own patch with a hostname, a link alias pinned to its MAC, a static address, and its install disk selected by serial. This is patches/nodes/talos-cp1.yaml; the other two differ only in the values:

yaml
apiVersion: v1alpha1
kind: HostnameConfig
hostname: talos-cp1
auto: off
---
apiVersion: v1alpha1
kind: LinkAliasConfig
name: net0
selector:
  match: mac(link.permanent_addr) == "3c:ec:ef:10:20:11"
---
apiVersion: v1alpha1
kind: LinkConfig
name: net0
addresses:
  - address: 10.10.20.11/24
routes:
  - gateway: 10.10.20.1
---
apiVersion: v1alpha1
kind: UnattendedInstallConfig
provisioning:
  diskSelector:
    match: disk.serial == "S6XXNX0T100011"

The alias is what lets controlplane.yaml refer to net0 on all three machines even if the kernel names the NIC differently on each: Talos treats a link alias as interchangeable with the physical name anywhere in the network configuration.

Why the VIP is for Kubernetes only

Two details from the VIP documentation shape the rest of the process. The VIP only comes alive after etcd is bootstrapped, because the election runs through etcd. And Sidero Labs warns not to use the VIP as an endpoint in talosconfig: if etcd or the API server is what's broken, the VIP is gone too, and you'd lose the Talos API access you need to fix it. So the Kubernetes endpoint is https://10.10.20.10:6443, while talosctl talks to the three node addresses directly. The failover is quick when the owner shuts down gracefully and takes up to about a minute after a crash or power loss, which is fine for kubectl and irrelevant to pods, since in-cluster clients reach the API server through KubePrism rather than the VIP.

Apply, then bootstrap exactly once

Applying configuration is a single talosctl apply-config --insecure per node, pointed at that node's maintenance-mode address with its generated file. The node validates the config, installs Talos to the selected disk, and reboots into the installed system. If the config is invalid it's rejected over the API with the reason, which is a better failure mode than a kubeadm join hanging on a firewall rule you forgot. I also run talosctl validate --mode metal --strict on every generated file first, because a typo in a CEL expression is cheaper to find on a laptop.

Applying configuration to all three nodes gets you three machines whose etcd services sit in a join loop, waiting for a peer. That's where talosctl bootstrap comes in. You run it against one control plane node, once. It tells that node to stop waiting and start etcd as a single-member cluster; the other two join once Kubernetes is up on the first. The docs put it in capitals: run it once, on a single node. Skip it and every etcd service stays stuck; Talos' troubleshooting guide lists "all etcd services are stuck in Pre state" with the fix "make sure that a single member was bootstrapped." Bootstrap is also the command you'd use later for disaster recovery, with --recover-from pointing at an etcd snapshot, which is one more reason to treat it as a deliberate act rather than something a retry loop fires twice.

Once it settles, talosctl kubeconfig pulls an admin kubeconfig from the node and talosctl health confirms etcd, the control plane components, and node readiness. Nodes show NotReady until the CNI (Flannel by default) is running, then flip to Ready.

The whole sequence as one script

This is the script I run from the repository root. It generates the talosconfig and one config per node, validates them, applies them, waits for the installed systems to answer on the authenticated API, bootstraps the first node, and waits for health. It assumes DHCP reservations hand each node the same address in maintenance mode that its LinkConfig makes permanent; if yours differ, keep a second map of maintenance addresses for step 4.

bash
#!/usr/bin/env bash
# bootstrap-cluster.sh
#
# Generates one machine config per node from secrets.yaml + patches, applies
# each config to a node in maintenance mode, bootstraps etcd once, then pulls
# a kubeconfig and waits for the cluster to report healthy.
#
# Needs: talosctl matching the Talos version on the ISO, the patches/ tree
# described above, and DHCP reservations that give each node the same address
# in maintenance mode that its LinkConfig patch assigns permanently.
set -euo pipefail

CLUSTER_NAME="lab"
ENDPOINT="https://10.10.20.10:6443"   # the Layer 2 VIP from patches/controlplane.yaml
TALOS_CONTRACT="v1.14"                 # keep fixed for reproducible configs
K8S_VERSION="1.37.0"
BOOTSTRAP_NODE="talos-cp1"
OUT="_out"                             # generated files: never commit this directory

declare -A NODES=(
    [talos-cp1]="10.10.20.11"
    [talos-cp2]="10.10.20.12"
    [talos-cp3]="10.10.20.13"
)

mkdir -p "$OUT"
export TALOSCONFIG="$OUT/talosconfig"

# 1. Cluster secrets: generated once, then stored somewhere safer than this repo.
if [[ ! -f secrets.yaml ]]; then
    talosctl gen secrets --output-file secrets.yaml
fi

# 2. The talosctl client config, with every control plane node as an endpoint.
talosctl gen config "$CLUSTER_NAME" "$ENDPOINT" \
    --with-secrets secrets.yaml \
    --talos-version "$TALOS_CONTRACT" \
    --output-types talosconfig \
    --output "$TALOSCONFIG" \
    --force
talosctl config endpoint "${NODES[@]}"

# 3. One controlplane config per node, validated before anything touches hardware.
for name in "${!NODES[@]}"; do
    talosctl gen config "$CLUSTER_NAME" "$ENDPOINT" \
        --with-secrets secrets.yaml \
        --kubernetes-version "$K8S_VERSION" \
        --talos-version "$TALOS_CONTRACT" \
        --config-patch @patches/common.yaml \
        --config-patch @patches/controlplane.yaml \
        --config-patch "@patches/nodes/${name}.yaml" \
        --output-types controlplane \
        --output "$OUT/${name}.yaml" \
        --force
    talosctl validate --config "$OUT/${name}.yaml" --mode metal --strict
done

# 4. Apply over the unauthenticated maintenance API. This is the only --insecure step.
for name in "${!NODES[@]}"; do
    echo "applying $OUT/${name}.yaml to ${NODES[$name]}"
    talosctl apply-config --insecure --nodes "${NODES[$name]}" --file "$OUT/${name}.yaml"
done

# 5. Wait for every node to install, reboot and answer on the authenticated API.
for name in "${!NODES[@]}"; do
    ip="${NODES[$name]}"
    for attempt in $(seq 1 60); do
        if talosctl --nodes "$ip" version >/dev/null 2>&1; then
            echo "$name ($ip) is up"
            break
        fi
        if [[ $attempt -eq 60 ]]; then
            echo "$name ($ip) did not come up after 10 minutes" >&2
            exit 1
        fi
        sleep 10
    done
done

# 6. Bootstrap etcd exactly once, on exactly one node.
talosctl bootstrap --nodes "${NODES[$BOOTSTRAP_NODE]}"

# 7. Kubeconfig to a separate file, then wait for the whole cluster.
talosctl kubeconfig "$OUT/kubeconfig" --nodes "${NODES[$BOOTSTRAP_NODE]}" --merge=false --force
talosctl health \
    --nodes "${NODES[$BOOTSTRAP_NODE]}" \
    --control-plane-nodes "$(IFS=,; echo "${NODES[*]}")" \
    --wait-timeout 15m

KUBECONFIG="$OUT/kubeconfig" kubectl get nodes

On a healthy run, the last lines look like this:

text
waiting for all k8s nodes to report ready: OK
waiting for all control plane static pods to be running: OK
waiting for all control plane components to be ready: OK
waiting for kube-proxy to report ready: OK
waiting for coredns to report ready: OK
waiting for all k8s nodes to report schedulable: OK
NAME        STATUS   ROLES           AGE   VERSION
talos-cp1   Ready    control-plane   6m    v1.37.0
talos-cp2   Ready    control-plane   5m    v1.37.0
talos-cp3   Ready    control-plane   5m    v1.37.0

Two follow-ups belong on day one rather than "later." Take the first etcd snapshot (talosctl -n 10.10.20.11 etcd snapshot db.snapshot) and schedule them; the disaster recovery guide recommends snapshots on a schedule so you can restore the latest one. And unplug the USB stick or unmount the ISO. The production notes suggest doing it before applying configs, so the installer can't mistake the boot media for the install disk; either way, the nodes should boot from their own disks from here on.

What immutability actually buys you here

The payoff isn't visible on day one. It shows up months later when a node needs to be reimaged after a hardware swap: boot the ISO, update the MAC and disk serial in that node's patch, regenerate, apply. The process is identical to the first bootstrap, because there was never any drift to reconcile. The inputs are the only source of truth, applied the same way every time. Compare that to a traditional bare-metal node where six months of ad hoc apt install and manual kubelet flag tweaks make a "clean rebuild" a multi-hour archaeology project. Talos trades the flexibility of a general-purpose OS for a much smaller, much more predictable surface, and for a cluster whose only job is running Kubernetes, that trade is an easy one to make.

Once the cluster is up, Python: Talos – Health-Checking a Kubernetes Cluster from a Cron Job covers keeping an eye on it, and Talos Linux: Upgrading a Cluster Without a Maintenance Window covers the first upgrade.

References