[{"data":1,"prerenderedAt":2781},["ShallowReactive",2],{"post:\u002F2026\u002F08\u002F26\u002Ftalos-linux-upgrading-a-cluster-without-a-maintenance-window\u002F":3},{"post":4,"newer":2730,"older":2742,"related":2753,"series":2779},{"id":5,"title":6,"body":7,"canonical":2711,"categories":2712,"date":2714,"description":2715,"extension":2716,"featured":2717,"hero":2718,"image":2718,"meta":2719,"navigation":519,"path":2720,"readingTime":211,"seo":2721,"series":2718,"seriesOrder":2718,"sites":2722,"source":2718,"stem":2723,"tags":2724,"updated":2727,"url":2728,"__hash__":2729},"blog\u002Fblog\u002F2026\u002F08\u002F26\u002Ftalos-linux-upgrading-a-cluster-without-a-maintenance-window.md","Talos Linux: Upgrading a Cluster Without a Maintenance Window",{"type":8,"value":9,"toc":2701},"minimark",[10,14,22,31,38,41,44,77,108,116,122,126,136,139,481,492,1631,1637,1641,1644,1647,1667,1673,1688,1692,1699,2480,2486,2490,2499,2502,2543,2552,2567,2571,2595,2604,2608,2697],[11,12,13],"p",{},"Asking for a maintenance window used to be the normal cost of doing an OS or Kubernetes upgrade. Talos Linux, combined with a cluster that's actually built to tolerate a node disappearing, changes that math enough that I no longer schedule downtime for routine version upgrades. I schedule a rolling operation that happens during business hours, with nobody outside the platform team aware it's happening. That took some deliberate groundwork, not just trusting the upgrade command.",[11,15,16,17,21],{},"The commands below are current as of Talos 1.14, which changed one thing every existing upgrade runbook needs to pick up: the ",[18,19,20],"code",{},"ghcr.io\u002Fsiderolabs\u002Finstaller"," image is no longer published, and installer images come from the Image Factory instead.",[23,24,26,27,30],"h2",{"id":25},"what-talosctl-upgrade-does-and-what-it-doesnt","What ",[18,28,29],{},"talosctl upgrade"," does, and what it doesn't",[11,32,33,34,37],{},"When a node receives an upgrade request, it cordons itself, drains its pods, stops its services, unmounts its filesystems, writes the new image, and sets the bootloader to boot the new version once. After it comes back and verifies itself, it makes that permanent, rejoins the cluster, and uncordons. Upgrades use an A-B scheme that keeps the previous kernel and OS image, so a node that fails to boot the new version falls back on its own. A node that boots fine but runs your workloads badly needs a manual ",[18,35,36],{},"talosctl rollback",".",[11,39,40],{},"On control plane nodes Talos adds a guard that matters a lot for no-downtime work: it refuses to upgrade a control plane node if doing so would cost etcd quorum, and if several control plane nodes are asked to upgrade at once it only lets one proceed at a time. The docs are equally clear about what that guard doesn't cover. Nothing stops you from sending near-simultaneous upgrades to every node, and software that keeps its own quorum (their example is Rook\u002FCeph) may not recover from more than one node rebooting at a time.",[11,42,43],{},"Two flag changes trip up older runbooks:",[45,46,47,64],"ul",{},[48,49,50,57,58,60,61,63],"li",{},[51,52,53,56],"strong",{},[18,54,55],{},"--preserve"," is gone."," Since Talos 1.8 the installer never wipes the system disk on upgrade, so the release notes describe ",[18,59,55],{}," as always set, and current ",[18,62,29],{}," doesn't have the flag.",[48,65,66,72,73,76],{},[51,67,68,71],{},[18,69,70],{},"--image"," should always be explicit."," It defaults to the installer for the ",[18,74,75],{},"talosctl"," version in use, with the empty schematic. Omit it on a node built from a schematic with system extensions and the node upgrades without them. Look up the node's schematic first; Image Factory appends it to the extension list:",[78,79,84],"pre",{"className":80,"code":81,"language":82,"meta":83,"style":83},"language-bash shiki shiki-themes github-dark","talosctl get extensions --nodes 10.10.20.11\n","bash","",[18,85,86],{"__ignoreMap":83},[87,88,91,94,98,101,105],"span",{"class":89,"line":90},"line",1,[87,92,75],{"class":93},"svObZ",[87,95,97],{"class":96},"sU2Wk"," get",[87,99,100],{"class":96}," extensions",[87,102,104],{"class":103},"sDLfK"," --nodes",[87,106,107],{"class":103}," 10.10.20.11\n",[78,109,114],{"className":110,"code":112,"language":113,"meta":83},[111],"language-text","NODE          NAMESPACE   TYPE              ID   VERSION   NAME        VERSION\n10.10.20.11   runtime     ExtensionStatus   0    1         schematic   376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba\n","text",[18,115,112],{"__ignoreMap":83},[11,117,118,119,37],{},"That ID (the \"vanilla\" schematic, in this example) becomes ",[18,120,121],{},"factory.talos.dev\u002Fmetal-installer\u002F\u003Cschematic-id>:\u003Cversion>",[23,123,125],{"id":124},"the-precondition-is-workload-placement-not-the-upgrade-tool","The precondition is workload placement, not the upgrade tool",[11,127,128,129,131,132,135],{},"Both ",[18,130,29],{}," (which cordons and drains a node before touching it) and ",[18,133,134],{},"talosctl upgrade-k8s"," are built to be run safely against a live cluster. None of that matters if the cluster's workloads can't survive a node leaving. A drain goes through the Kubernetes Eviction API, which honours PodDisruptionBudgets, so an upgrade tool that drains correctly is necessary but not sufficient: if three replicas of a service all sit on the node being drained, the graceful drain just makes the outage orderly instead of preventing it.",[11,137,138],{},"For anything customer-facing, that means more than one replica, a PDB that reflects real availability requirements, and a spread constraint that keeps replicas on different nodes:",[78,140,144],{"className":141,"code":142,"language":143,"meta":83,"style":83},"language-yaml shiki shiki-themes github-dark","apiVersion: policy\u002Fv1\nkind: PodDisruptionBudget\nmetadata:\n  name: invoice-api\n  namespace: billing\nspec:\n  maxUnavailable: 1\n  selector:\n    matchLabels:\n      app: invoice-api\n---\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: invoice-api\n  namespace: billing\nspec:\n  replicas: 3\n  selector:\n    matchLabels:\n      app: invoice-api\n  template:\n    metadata:\n      labels:\n        app: invoice-api\n    spec:\n      topologySpreadConstraints:\n        - maxSkew: 1\n          topologyKey: kubernetes.io\u002Fhostname\n          whenUnsatisfiable: DoNotSchedule\n          labelSelector:\n            matchLabels:\n              app: invoice-api\n      containers:\n        - name: api\n          image: registry.example.com\u002Fbilling\u002Finvoice-api:2.4.1\n","yaml",[18,145,146,159,170,179,190,201,209,220,228,236,246,252,262,272,279,288,297,304,315,322,329,338,346,354,362,372,380,388,401,412,423,431,439,449,457,470],{"__ignoreMap":83},[87,147,148,152,156],{"class":89,"line":90},[87,149,151],{"class":150},"s4JwU","apiVersion",[87,153,155],{"class":154},"s95oV",": ",[87,157,158],{"class":96},"policy\u002Fv1\n",[87,160,162,165,167],{"class":89,"line":161},2,[87,163,164],{"class":150},"kind",[87,166,155],{"class":154},[87,168,169],{"class":96},"PodDisruptionBudget\n",[87,171,173,176],{"class":89,"line":172},3,[87,174,175],{"class":150},"metadata",[87,177,178],{"class":154},":\n",[87,180,182,185,187],{"class":89,"line":181},4,[87,183,184],{"class":150},"  name",[87,186,155],{"class":154},[87,188,189],{"class":96},"invoice-api\n",[87,191,193,196,198],{"class":89,"line":192},5,[87,194,195],{"class":150},"  namespace",[87,197,155],{"class":154},[87,199,200],{"class":96},"billing\n",[87,202,204,207],{"class":89,"line":203},6,[87,205,206],{"class":150},"spec",[87,208,178],{"class":154},[87,210,212,215,217],{"class":89,"line":211},7,[87,213,214],{"class":150},"  maxUnavailable",[87,216,155],{"class":154},[87,218,219],{"class":103},"1\n",[87,221,223,226],{"class":89,"line":222},8,[87,224,225],{"class":150},"  selector",[87,227,178],{"class":154},[87,229,231,234],{"class":89,"line":230},9,[87,232,233],{"class":150},"    matchLabels",[87,235,178],{"class":154},[87,237,239,242,244],{"class":89,"line":238},10,[87,240,241],{"class":150},"      app",[87,243,155],{"class":154},[87,245,189],{"class":96},[87,247,249],{"class":89,"line":248},11,[87,250,251],{"class":93},"---\n",[87,253,255,257,259],{"class":89,"line":254},12,[87,256,151],{"class":150},[87,258,155],{"class":154},[87,260,261],{"class":96},"apps\u002Fv1\n",[87,263,265,267,269],{"class":89,"line":264},13,[87,266,164],{"class":150},[87,268,155],{"class":154},[87,270,271],{"class":96},"Deployment\n",[87,273,275,277],{"class":89,"line":274},14,[87,276,175],{"class":150},[87,278,178],{"class":154},[87,280,282,284,286],{"class":89,"line":281},15,[87,283,184],{"class":150},[87,285,155],{"class":154},[87,287,189],{"class":96},[87,289,291,293,295],{"class":89,"line":290},16,[87,292,195],{"class":150},[87,294,155],{"class":154},[87,296,200],{"class":96},[87,298,300,302],{"class":89,"line":299},17,[87,301,206],{"class":150},[87,303,178],{"class":154},[87,305,307,310,312],{"class":89,"line":306},18,[87,308,309],{"class":150},"  replicas",[87,311,155],{"class":154},[87,313,314],{"class":103},"3\n",[87,316,318,320],{"class":89,"line":317},19,[87,319,225],{"class":150},[87,321,178],{"class":154},[87,323,325,327],{"class":89,"line":324},20,[87,326,233],{"class":150},[87,328,178],{"class":154},[87,330,332,334,336],{"class":89,"line":331},21,[87,333,241],{"class":150},[87,335,155],{"class":154},[87,337,189],{"class":96},[87,339,341,344],{"class":89,"line":340},22,[87,342,343],{"class":150},"  template",[87,345,178],{"class":154},[87,347,349,352],{"class":89,"line":348},23,[87,350,351],{"class":150},"    metadata",[87,353,178],{"class":154},[87,355,357,360],{"class":89,"line":356},24,[87,358,359],{"class":150},"      labels",[87,361,178],{"class":154},[87,363,365,368,370],{"class":89,"line":364},25,[87,366,367],{"class":150},"        app",[87,369,155],{"class":154},[87,371,189],{"class":96},[87,373,375,378],{"class":89,"line":374},26,[87,376,377],{"class":150},"    spec",[87,379,178],{"class":154},[87,381,383,386],{"class":89,"line":382},27,[87,384,385],{"class":150},"      topologySpreadConstraints",[87,387,178],{"class":154},[87,389,391,394,397,399],{"class":89,"line":390},28,[87,392,393],{"class":154},"        - ",[87,395,396],{"class":150},"maxSkew",[87,398,155],{"class":154},[87,400,219],{"class":103},[87,402,404,407,409],{"class":89,"line":403},29,[87,405,406],{"class":150},"          topologyKey",[87,408,155],{"class":154},[87,410,411],{"class":96},"kubernetes.io\u002Fhostname\n",[87,413,415,418,420],{"class":89,"line":414},30,[87,416,417],{"class":150},"          whenUnsatisfiable",[87,419,155],{"class":154},[87,421,422],{"class":96},"DoNotSchedule\n",[87,424,426,429],{"class":89,"line":425},31,[87,427,428],{"class":150},"          labelSelector",[87,430,178],{"class":154},[87,432,434,437],{"class":89,"line":433},32,[87,435,436],{"class":150},"            matchLabels",[87,438,178],{"class":154},[87,440,442,445,447],{"class":89,"line":441},33,[87,443,444],{"class":150},"              app",[87,446,155],{"class":154},[87,448,189],{"class":96},[87,450,452,455],{"class":89,"line":451},34,[87,453,454],{"class":150},"      containers",[87,456,178],{"class":154},[87,458,460,462,465,467],{"class":89,"line":459},35,[87,461,393],{"class":154},[87,463,464],{"class":150},"name",[87,466,155],{"class":154},[87,468,469],{"class":96},"api\n",[87,471,473,476,478],{"class":89,"line":472},36,[87,474,475],{"class":150},"          image",[87,477,155],{"class":154},[87,479,480],{"class":96},"registry.example.com\u002Fbilling\u002Finvoice-api:2.4.1\n",[11,482,483,484,487,488,491],{},"The mirror-image mistake is a PDB that can never be satisfied, such as ",[18,485,486],{},"minAvailable: 1"," on a single-replica Deployment. That PDB permanently allows zero disruptions, and the drain waits on it until ",[18,489,490],{},"--drain-timeout"," (five minutes by default) runs out. So before every rolling operation I run a preflight that answers one question: can this cluster lose one node right now?",[78,493,497],{"className":494,"code":495,"language":496,"meta":83,"style":83},"language-python shiki shiki-themes github-dark","#!\u002Fusr\u002Fbin\u002Fenv python3\n\"\"\"\nupgrade_preflight.py\n\nAnswers \"can this cluster lose one node right now?\" before a rolling Talos or\nKubernetes upgrade. Blocks (exit 1) on:\n  - nodes that are not Ready or are already cordoned\n  - PodDisruptionBudgets that currently allow zero disruptions (a drain will stall)\n  - Deployments\u002FStatefulSets outside the ignored namespaces with fewer than 2 replicas\n  - multi-replica Deployments whose Ready pods all sit on one node\n\nRequires: Python 3.10+, `pip install kubernetes`, a kubeconfig that can list\nnodes, pods, deployments, statefulsets and poddisruptionbudgets.\n\"\"\"\n\nimport argparse\nimport sys\nfrom collections import defaultdict\n\nfrom kubernetes import client, config\n\n\ndef main():\n    parser = argparse.ArgumentParser(description=\"Pre-flight checks before a rolling node upgrade.\")\n    parser.add_argument(\"--kubeconfig\", default=None, help=\"Path to kubeconfig (default: standard lookup)\")\n    parser.add_argument(\"--ignore-namespace\", action=\"append\", default=[\"kube-system\"],\n                        help=\"Namespace to skip in the workload checks (repeatable; kube-system is skipped by default)\")\n    args = parser.parse_args()\n\n    config.load_kube_config(config_file=args.kubeconfig)\n    core, apps, policy = client.CoreV1Api(), client.AppsV1Api(), client.PolicyV1Api()\n    ignored = set(args.ignore_namespace)\n    blockers = []\n\n    for node in core.list_node().items:\n        ready = next((c for c in node.status.conditions or [] if c.type == \"Ready\"), None)\n        if ready is None or ready.status != \"True\":\n            blockers.append(f\"node {node.metadata.name} is not Ready\")\n        if node.spec.unschedulable:\n            blockers.append(f\"node {node.metadata.name} is already cordoned\")\n\n    for pdb in policy.list_pod_disruption_budget_for_all_namespaces().items:\n        if pdb.metadata.namespace in ignored:\n            continue\n        if (pdb.status.disruptions_allowed or 0) \u003C 1:\n            blockers.append(\n                f\"PDB {pdb.metadata.namespace}\u002F{pdb.metadata.name} allows 0 disruptions \"\n                f\"(healthy {pdb.status.current_healthy}\u002F{pdb.status.desired_healthy} desired)\"\n            )\n\n    workloads = [(\"Deployment\", d) for d in apps.list_deployment_for_all_namespaces().items]\n    workloads += [(\"StatefulSet\", s) for s in apps.list_stateful_set_for_all_namespaces().items]\n    for kind, obj in workloads:\n        ns, name = obj.metadata.namespace, obj.metadata.name\n        replicas = obj.spec.replicas if obj.spec.replicas is not None else 1\n        if ns not in ignored and 0 \u003C replicas \u003C 2:\n            blockers.append(f\"{kind} {ns}\u002F{name} has {replicas} replica: it will go down while its node drains\")\n\n    # Ready pods per ReplicaSet owner, grouped by node: catches \"3 replicas, 1 node\".\n    placement = defaultdict(set)\n    for pod in core.list_pod_for_all_namespaces(field_selector=\"status.phase=Running\").items:\n        if pod.metadata.namespace in ignored or not pod.spec.node_name:\n            continue\n        for owner in pod.metadata.owner_references or []:\n            if owner.kind == \"ReplicaSet\":\n                placement[(pod.metadata.namespace, owner.name)].add(pod.spec.node_name)\n    for (ns, rs), nodes in placement.items():\n        replica_set = apps.read_namespaced_replica_set(rs, ns)\n        if (replica_set.spec.replicas or 0) >= 2 and len(nodes) == 1:\n            blockers.append(f\"ReplicaSet {ns}\u002F{rs}: all {replica_set.spec.replicas} replicas run on {next(iter(nodes))}\")\n\n    if blockers:\n        print(\"NOT SAFE to take a node out:\")\n        for item in blockers:\n            print(f\"  - {item}\")\n        sys.exit(1)\n\n    print(\"OK: every node Ready, no PDB at zero, no single-replica or single-node workloads.\")\n    sys.exit(0)\n\n\nif __name__ == \"__main__\":\n    main()\n","python",[18,498,499,505,510,515,521,526,531,536,541,546,551,555,560,565,569,573,582,589,602,606,618,622,626,637,660,691,723,735,745,749,762,772,785,795,799,813,862,891,917,925,945,950,963,976,982,1006,1012,1041,1068,1074,1079,1106,1132,1145,1156,1184,1219,1266,1271,1277,1293,1317,1336,1341,1360,1376,1382,1395,1406,1440,1496,1501,1510,1523,1535,1559,1570,1575,1588,1599,1604,1609,1625],{"__ignoreMap":83},[87,500,501],{"class":89,"line":90},[87,502,504],{"class":503},"sAwPA","#!\u002Fusr\u002Fbin\u002Fenv python3\n",[87,506,507],{"class":89,"line":161},[87,508,509],{"class":96},"\"\"\"\n",[87,511,512],{"class":89,"line":172},[87,513,514],{"class":96},"upgrade_preflight.py\n",[87,516,517],{"class":89,"line":181},[87,518,520],{"emptyLinePlaceholder":519},true,"\n",[87,522,523],{"class":89,"line":192},[87,524,525],{"class":96},"Answers \"can this cluster lose one node right now?\" before a rolling Talos or\n",[87,527,528],{"class":89,"line":203},[87,529,530],{"class":96},"Kubernetes upgrade. Blocks (exit 1) on:\n",[87,532,533],{"class":89,"line":211},[87,534,535],{"class":96},"  - nodes that are not Ready or are already cordoned\n",[87,537,538],{"class":89,"line":222},[87,539,540],{"class":96},"  - PodDisruptionBudgets that currently allow zero disruptions (a drain will stall)\n",[87,542,543],{"class":89,"line":230},[87,544,545],{"class":96},"  - Deployments\u002FStatefulSets outside the ignored namespaces with fewer than 2 replicas\n",[87,547,548],{"class":89,"line":238},[87,549,550],{"class":96},"  - multi-replica Deployments whose Ready pods all sit on one node\n",[87,552,553],{"class":89,"line":248},[87,554,520],{"emptyLinePlaceholder":519},[87,556,557],{"class":89,"line":254},[87,558,559],{"class":96},"Requires: Python 3.10+, `pip install kubernetes`, a kubeconfig that can list\n",[87,561,562],{"class":89,"line":264},[87,563,564],{"class":96},"nodes, pods, deployments, statefulsets and poddisruptionbudgets.\n",[87,566,567],{"class":89,"line":274},[87,568,509],{"class":96},[87,570,571],{"class":89,"line":281},[87,572,520],{"emptyLinePlaceholder":519},[87,574,575,579],{"class":89,"line":290},[87,576,578],{"class":577},"snl16","import",[87,580,581],{"class":154}," argparse\n",[87,583,584,586],{"class":89,"line":299},[87,585,578],{"class":577},[87,587,588],{"class":154}," sys\n",[87,590,591,594,597,599],{"class":89,"line":306},[87,592,593],{"class":577},"from",[87,595,596],{"class":154}," collections ",[87,598,578],{"class":577},[87,600,601],{"class":154}," defaultdict\n",[87,603,604],{"class":89,"line":317},[87,605,520],{"emptyLinePlaceholder":519},[87,607,608,610,613,615],{"class":89,"line":324},[87,609,593],{"class":577},[87,611,612],{"class":154}," kubernetes ",[87,614,578],{"class":577},[87,616,617],{"class":154}," client, config\n",[87,619,620],{"class":89,"line":331},[87,621,520],{"emptyLinePlaceholder":519},[87,623,624],{"class":89,"line":340},[87,625,520],{"emptyLinePlaceholder":519},[87,627,628,631,634],{"class":89,"line":348},[87,629,630],{"class":577},"def",[87,632,633],{"class":93}," main",[87,635,636],{"class":154},"():\n",[87,638,639,642,645,648,652,654,657],{"class":89,"line":356},[87,640,641],{"class":154},"    parser ",[87,643,644],{"class":577},"=",[87,646,647],{"class":154}," argparse.ArgumentParser(",[87,649,651],{"class":650},"s9osk","description",[87,653,644],{"class":577},[87,655,656],{"class":96},"\"Pre-flight checks before a rolling node upgrade.\"",[87,658,659],{"class":154},")\n",[87,661,662,665,668,671,674,676,679,681,684,686,689],{"class":89,"line":364},[87,663,664],{"class":154},"    parser.add_argument(",[87,666,667],{"class":96},"\"--kubeconfig\"",[87,669,670],{"class":154},", ",[87,672,673],{"class":650},"default",[87,675,644],{"class":577},[87,677,678],{"class":103},"None",[87,680,670],{"class":154},[87,682,683],{"class":650},"help",[87,685,644],{"class":577},[87,687,688],{"class":96},"\"Path to kubeconfig (default: standard lookup)\"",[87,690,659],{"class":154},[87,692,693,695,698,700,703,705,708,710,712,714,717,720],{"class":89,"line":374},[87,694,664],{"class":154},[87,696,697],{"class":96},"\"--ignore-namespace\"",[87,699,670],{"class":154},[87,701,702],{"class":650},"action",[87,704,644],{"class":577},[87,706,707],{"class":96},"\"append\"",[87,709,670],{"class":154},[87,711,673],{"class":650},[87,713,644],{"class":577},[87,715,716],{"class":154},"[",[87,718,719],{"class":96},"\"kube-system\"",[87,721,722],{"class":154},"],\n",[87,724,725,728,730,733],{"class":89,"line":382},[87,726,727],{"class":650},"                        help",[87,729,644],{"class":577},[87,731,732],{"class":96},"\"Namespace to skip in the workload checks (repeatable; kube-system is skipped by default)\"",[87,734,659],{"class":154},[87,736,737,740,742],{"class":89,"line":390},[87,738,739],{"class":154},"    args ",[87,741,644],{"class":577},[87,743,744],{"class":154}," parser.parse_args()\n",[87,746,747],{"class":89,"line":403},[87,748,520],{"emptyLinePlaceholder":519},[87,750,751,754,757,759],{"class":89,"line":414},[87,752,753],{"class":154},"    config.load_kube_config(",[87,755,756],{"class":650},"config_file",[87,758,644],{"class":577},[87,760,761],{"class":154},"args.kubeconfig)\n",[87,763,764,767,769],{"class":89,"line":425},[87,765,766],{"class":154},"    core, apps, policy ",[87,768,644],{"class":577},[87,770,771],{"class":154}," client.CoreV1Api(), client.AppsV1Api(), client.PolicyV1Api()\n",[87,773,774,777,779,782],{"class":89,"line":433},[87,775,776],{"class":154},"    ignored ",[87,778,644],{"class":577},[87,780,781],{"class":103}," set",[87,783,784],{"class":154},"(args.ignore_namespace)\n",[87,786,787,790,792],{"class":89,"line":441},[87,788,789],{"class":154},"    blockers ",[87,791,644],{"class":577},[87,793,794],{"class":154}," []\n",[87,796,797],{"class":89,"line":451},[87,798,520],{"emptyLinePlaceholder":519},[87,800,801,804,807,810],{"class":89,"line":459},[87,802,803],{"class":577},"    for",[87,805,806],{"class":154}," node ",[87,808,809],{"class":577},"in",[87,811,812],{"class":154}," core.list_node().items:\n",[87,814,815,818,820,823,826,829,832,834,837,840,843,846,849,852,855,858,860],{"class":89,"line":472},[87,816,817],{"class":154},"        ready ",[87,819,644],{"class":577},[87,821,822],{"class":103}," next",[87,824,825],{"class":154},"((c ",[87,827,828],{"class":577},"for",[87,830,831],{"class":154}," c ",[87,833,809],{"class":577},[87,835,836],{"class":154}," node.status.conditions ",[87,838,839],{"class":577},"or",[87,841,842],{"class":154}," [] ",[87,844,845],{"class":577},"if",[87,847,848],{"class":154}," c.type ",[87,850,851],{"class":577},"==",[87,853,854],{"class":96}," \"Ready\"",[87,856,857],{"class":154},"), ",[87,859,678],{"class":103},[87,861,659],{"class":154},[87,863,865,868,871,874,877,880,883,886,889],{"class":89,"line":864},37,[87,866,867],{"class":577},"        if",[87,869,870],{"class":154}," ready ",[87,872,873],{"class":577},"is",[87,875,876],{"class":103}," None",[87,878,879],{"class":577}," or",[87,881,882],{"class":154}," ready.status ",[87,884,885],{"class":577},"!=",[87,887,888],{"class":96}," \"True\"",[87,890,178],{"class":154},[87,892,894,897,900,903,906,909,912,915],{"class":89,"line":893},38,[87,895,896],{"class":154},"            blockers.append(",[87,898,899],{"class":577},"f",[87,901,902],{"class":96},"\"node ",[87,904,905],{"class":103},"{",[87,907,908],{"class":154},"node.metadata.name",[87,910,911],{"class":103},"}",[87,913,914],{"class":96}," is not Ready\"",[87,916,659],{"class":154},[87,918,920,922],{"class":89,"line":919},39,[87,921,867],{"class":577},[87,923,924],{"class":154}," node.spec.unschedulable:\n",[87,926,928,930,932,934,936,938,940,943],{"class":89,"line":927},40,[87,929,896],{"class":154},[87,931,899],{"class":577},[87,933,902],{"class":96},[87,935,905],{"class":103},[87,937,908],{"class":154},[87,939,911],{"class":103},[87,941,942],{"class":96}," is already cordoned\"",[87,944,659],{"class":154},[87,946,948],{"class":89,"line":947},41,[87,949,520],{"emptyLinePlaceholder":519},[87,951,953,955,958,960],{"class":89,"line":952},42,[87,954,803],{"class":577},[87,956,957],{"class":154}," pdb ",[87,959,809],{"class":577},[87,961,962],{"class":154}," policy.list_pod_disruption_budget_for_all_namespaces().items:\n",[87,964,966,968,971,973],{"class":89,"line":965},43,[87,967,867],{"class":577},[87,969,970],{"class":154}," pdb.metadata.namespace ",[87,972,809],{"class":577},[87,974,975],{"class":154}," ignored:\n",[87,977,979],{"class":89,"line":978},44,[87,980,981],{"class":577},"            continue\n",[87,983,985,987,990,992,995,998,1001,1004],{"class":89,"line":984},45,[87,986,867],{"class":577},[87,988,989],{"class":154}," (pdb.status.disruptions_allowed ",[87,991,839],{"class":577},[87,993,994],{"class":103}," 0",[87,996,997],{"class":154},") ",[87,999,1000],{"class":577},"\u003C",[87,1002,1003],{"class":103}," 1",[87,1005,178],{"class":154},[87,1007,1009],{"class":89,"line":1008},46,[87,1010,1011],{"class":154},"            blockers.append(\n",[87,1013,1015,1018,1021,1023,1026,1028,1031,1033,1036,1038],{"class":89,"line":1014},47,[87,1016,1017],{"class":577},"                f",[87,1019,1020],{"class":96},"\"PDB ",[87,1022,905],{"class":103},[87,1024,1025],{"class":154},"pdb.metadata.namespace",[87,1027,911],{"class":103},[87,1029,1030],{"class":96},"\u002F",[87,1032,905],{"class":103},[87,1034,1035],{"class":154},"pdb.metadata.name",[87,1037,911],{"class":103},[87,1039,1040],{"class":96}," allows 0 disruptions \"\n",[87,1042,1044,1046,1049,1051,1054,1056,1058,1060,1063,1065],{"class":89,"line":1043},48,[87,1045,1017],{"class":577},[87,1047,1048],{"class":96},"\"(healthy ",[87,1050,905],{"class":103},[87,1052,1053],{"class":154},"pdb.status.current_healthy",[87,1055,911],{"class":103},[87,1057,1030],{"class":96},[87,1059,905],{"class":103},[87,1061,1062],{"class":154},"pdb.status.desired_healthy",[87,1064,911],{"class":103},[87,1066,1067],{"class":96}," desired)\"\n",[87,1069,1071],{"class":89,"line":1070},49,[87,1072,1073],{"class":154},"            )\n",[87,1075,1077],{"class":89,"line":1076},50,[87,1078,520],{"emptyLinePlaceholder":519},[87,1080,1082,1085,1087,1090,1093,1096,1098,1101,1103],{"class":89,"line":1081},51,[87,1083,1084],{"class":154},"    workloads ",[87,1086,644],{"class":577},[87,1088,1089],{"class":154}," [(",[87,1091,1092],{"class":96},"\"Deployment\"",[87,1094,1095],{"class":154},", d) ",[87,1097,828],{"class":577},[87,1099,1100],{"class":154}," d ",[87,1102,809],{"class":577},[87,1104,1105],{"class":154}," apps.list_deployment_for_all_namespaces().items]\n",[87,1107,1109,1111,1114,1116,1119,1122,1124,1127,1129],{"class":89,"line":1108},52,[87,1110,1084],{"class":154},[87,1112,1113],{"class":577},"+=",[87,1115,1089],{"class":154},[87,1117,1118],{"class":96},"\"StatefulSet\"",[87,1120,1121],{"class":154},", s) ",[87,1123,828],{"class":577},[87,1125,1126],{"class":154}," s ",[87,1128,809],{"class":577},[87,1130,1131],{"class":154}," apps.list_stateful_set_for_all_namespaces().items]\n",[87,1133,1135,1137,1140,1142],{"class":89,"line":1134},53,[87,1136,803],{"class":577},[87,1138,1139],{"class":154}," kind, obj ",[87,1141,809],{"class":577},[87,1143,1144],{"class":154}," workloads:\n",[87,1146,1148,1151,1153],{"class":89,"line":1147},54,[87,1149,1150],{"class":154},"        ns, name ",[87,1152,644],{"class":577},[87,1154,1155],{"class":154}," obj.metadata.namespace, obj.metadata.name\n",[87,1157,1159,1162,1164,1167,1169,1171,1173,1176,1178,1181],{"class":89,"line":1158},55,[87,1160,1161],{"class":154},"        replicas ",[87,1163,644],{"class":577},[87,1165,1166],{"class":154}," obj.spec.replicas ",[87,1168,845],{"class":577},[87,1170,1166],{"class":154},[87,1172,873],{"class":577},[87,1174,1175],{"class":577}," not",[87,1177,876],{"class":103},[87,1179,1180],{"class":577}," else",[87,1182,1183],{"class":103}," 1\n",[87,1185,1187,1189,1192,1195,1198,1201,1204,1206,1209,1212,1214,1217],{"class":89,"line":1186},56,[87,1188,867],{"class":577},[87,1190,1191],{"class":154}," ns ",[87,1193,1194],{"class":577},"not",[87,1196,1197],{"class":577}," in",[87,1199,1200],{"class":154}," ignored ",[87,1202,1203],{"class":577},"and",[87,1205,994],{"class":103},[87,1207,1208],{"class":577}," \u003C",[87,1210,1211],{"class":154}," replicas ",[87,1213,1000],{"class":577},[87,1215,1216],{"class":103}," 2",[87,1218,178],{"class":154},[87,1220,1222,1224,1226,1229,1231,1233,1235,1238,1241,1243,1245,1247,1249,1251,1254,1256,1259,1261,1264],{"class":89,"line":1221},57,[87,1223,896],{"class":154},[87,1225,899],{"class":577},[87,1227,1228],{"class":96},"\"",[87,1230,905],{"class":103},[87,1232,164],{"class":154},[87,1234,911],{"class":103},[87,1236,1237],{"class":103}," {",[87,1239,1240],{"class":154},"ns",[87,1242,911],{"class":103},[87,1244,1030],{"class":96},[87,1246,905],{"class":103},[87,1248,464],{"class":154},[87,1250,911],{"class":103},[87,1252,1253],{"class":96}," has ",[87,1255,905],{"class":103},[87,1257,1258],{"class":154},"replicas",[87,1260,911],{"class":103},[87,1262,1263],{"class":96}," replica: it will go down while its node drains\"",[87,1265,659],{"class":154},[87,1267,1269],{"class":89,"line":1268},58,[87,1270,520],{"emptyLinePlaceholder":519},[87,1272,1274],{"class":89,"line":1273},59,[87,1275,1276],{"class":503},"    # Ready pods per ReplicaSet owner, grouped by node: catches \"3 replicas, 1 node\".\n",[87,1278,1280,1283,1285,1288,1291],{"class":89,"line":1279},60,[87,1281,1282],{"class":154},"    placement ",[87,1284,644],{"class":577},[87,1286,1287],{"class":154}," defaultdict(",[87,1289,1290],{"class":103},"set",[87,1292,659],{"class":154},[87,1294,1296,1298,1301,1303,1306,1309,1311,1314],{"class":89,"line":1295},61,[87,1297,803],{"class":577},[87,1299,1300],{"class":154}," pod ",[87,1302,809],{"class":577},[87,1304,1305],{"class":154}," core.list_pod_for_all_namespaces(",[87,1307,1308],{"class":650},"field_selector",[87,1310,644],{"class":577},[87,1312,1313],{"class":96},"\"status.phase=Running\"",[87,1315,1316],{"class":154},").items:\n",[87,1318,1320,1322,1325,1327,1329,1331,1333],{"class":89,"line":1319},62,[87,1321,867],{"class":577},[87,1323,1324],{"class":154}," pod.metadata.namespace ",[87,1326,809],{"class":577},[87,1328,1200],{"class":154},[87,1330,839],{"class":577},[87,1332,1175],{"class":577},[87,1334,1335],{"class":154}," pod.spec.node_name:\n",[87,1337,1339],{"class":89,"line":1338},63,[87,1340,981],{"class":577},[87,1342,1344,1347,1350,1352,1355,1357],{"class":89,"line":1343},64,[87,1345,1346],{"class":577},"        for",[87,1348,1349],{"class":154}," owner ",[87,1351,809],{"class":577},[87,1353,1354],{"class":154}," pod.metadata.owner_references ",[87,1356,839],{"class":577},[87,1358,1359],{"class":154}," []:\n",[87,1361,1363,1366,1369,1371,1374],{"class":89,"line":1362},65,[87,1364,1365],{"class":577},"            if",[87,1367,1368],{"class":154}," owner.kind ",[87,1370,851],{"class":577},[87,1372,1373],{"class":96}," \"ReplicaSet\"",[87,1375,178],{"class":154},[87,1377,1379],{"class":89,"line":1378},66,[87,1380,1381],{"class":154},"                placement[(pod.metadata.namespace, owner.name)].add(pod.spec.node_name)\n",[87,1383,1385,1387,1390,1392],{"class":89,"line":1384},67,[87,1386,803],{"class":577},[87,1388,1389],{"class":154}," (ns, rs), nodes ",[87,1391,809],{"class":577},[87,1393,1394],{"class":154}," placement.items():\n",[87,1396,1398,1401,1403],{"class":89,"line":1397},68,[87,1399,1400],{"class":154},"        replica_set ",[87,1402,644],{"class":577},[87,1404,1405],{"class":154}," apps.read_namespaced_replica_set(rs, ns)\n",[87,1407,1409,1411,1414,1416,1418,1420,1423,1425,1428,1431,1434,1436,1438],{"class":89,"line":1408},69,[87,1410,867],{"class":577},[87,1412,1413],{"class":154}," (replica_set.spec.replicas ",[87,1415,839],{"class":577},[87,1417,994],{"class":103},[87,1419,997],{"class":154},[87,1421,1422],{"class":577},">=",[87,1424,1216],{"class":103},[87,1426,1427],{"class":577}," and",[87,1429,1430],{"class":103}," len",[87,1432,1433],{"class":154},"(nodes) ",[87,1435,851],{"class":577},[87,1437,1003],{"class":103},[87,1439,178],{"class":154},[87,1441,1443,1445,1447,1450,1452,1454,1456,1458,1460,1463,1465,1468,1470,1473,1475,1478,1481,1484,1487,1490,1492,1494],{"class":89,"line":1442},70,[87,1444,896],{"class":154},[87,1446,899],{"class":577},[87,1448,1449],{"class":96},"\"ReplicaSet ",[87,1451,905],{"class":103},[87,1453,1240],{"class":154},[87,1455,911],{"class":103},[87,1457,1030],{"class":96},[87,1459,905],{"class":103},[87,1461,1462],{"class":154},"rs",[87,1464,911],{"class":103},[87,1466,1467],{"class":96},": all ",[87,1469,905],{"class":103},[87,1471,1472],{"class":154},"replica_set.spec.replicas",[87,1474,911],{"class":103},[87,1476,1477],{"class":96}," replicas run on ",[87,1479,1480],{"class":103},"{next",[87,1482,1483],{"class":154},"(",[87,1485,1486],{"class":103},"iter",[87,1488,1489],{"class":154},"(nodes))",[87,1491,911],{"class":103},[87,1493,1228],{"class":96},[87,1495,659],{"class":154},[87,1497,1499],{"class":89,"line":1498},71,[87,1500,520],{"emptyLinePlaceholder":519},[87,1502,1504,1507],{"class":89,"line":1503},72,[87,1505,1506],{"class":577},"    if",[87,1508,1509],{"class":154}," blockers:\n",[87,1511,1513,1516,1518,1521],{"class":89,"line":1512},73,[87,1514,1515],{"class":103},"        print",[87,1517,1483],{"class":154},[87,1519,1520],{"class":96},"\"NOT SAFE to take a node out:\"",[87,1522,659],{"class":154},[87,1524,1526,1528,1531,1533],{"class":89,"line":1525},74,[87,1527,1346],{"class":577},[87,1529,1530],{"class":154}," item ",[87,1532,809],{"class":577},[87,1534,1509],{"class":154},[87,1536,1538,1541,1543,1545,1548,1550,1553,1555,1557],{"class":89,"line":1537},75,[87,1539,1540],{"class":103},"            print",[87,1542,1483],{"class":154},[87,1544,899],{"class":577},[87,1546,1547],{"class":96},"\"  - ",[87,1549,905],{"class":103},[87,1551,1552],{"class":154},"item",[87,1554,911],{"class":103},[87,1556,1228],{"class":96},[87,1558,659],{"class":154},[87,1560,1562,1565,1568],{"class":89,"line":1561},76,[87,1563,1564],{"class":154},"        sys.exit(",[87,1566,1567],{"class":103},"1",[87,1569,659],{"class":154},[87,1571,1573],{"class":89,"line":1572},77,[87,1574,520],{"emptyLinePlaceholder":519},[87,1576,1578,1581,1583,1586],{"class":89,"line":1577},78,[87,1579,1580],{"class":103},"    print",[87,1582,1483],{"class":154},[87,1584,1585],{"class":96},"\"OK: every node Ready, no PDB at zero, no single-replica or single-node workloads.\"",[87,1587,659],{"class":154},[87,1589,1591,1594,1597],{"class":89,"line":1590},79,[87,1592,1593],{"class":154},"    sys.exit(",[87,1595,1596],{"class":103},"0",[87,1598,659],{"class":154},[87,1600,1602],{"class":89,"line":1601},80,[87,1603,520],{"emptyLinePlaceholder":519},[87,1605,1607],{"class":89,"line":1606},81,[87,1608,520],{"emptyLinePlaceholder":519},[87,1610,1612,1614,1617,1620,1623],{"class":89,"line":1611},82,[87,1613,845],{"class":577},[87,1615,1616],{"class":103}," __name__",[87,1618,1619],{"class":577}," ==",[87,1621,1622],{"class":96}," \"__main__\"",[87,1624,178],{"class":154},[87,1626,1628],{"class":89,"line":1627},83,[87,1629,1630],{"class":154},"    main()\n",[78,1632,1635],{"className":1633,"code":1634,"language":113,"meta":83},[111],"NOT SAFE to take a node out:\n  - PDB billing\u002Finvoice-worker allows 0 disruptions (healthy 1\u002F1 desired)\n  - Deployment billing\u002Finvoice-worker has 1 replica: it will go down while its node drains\n",[18,1636,1634],{"__ignoreMap":83},[23,1638,1640],{"id":1639},"etcd-is-the-part-that-actually-requires-care","Etcd is the part that actually requires care",[11,1642,1643],{},"Worker nodes are the easy case; losing one briefly is exactly what the scheduler and PodDisruptionBudgets exist to absorb. Control plane nodes running etcd are where I slow down, because etcd's quorum math is unforgiving. Talos' disaster recovery guide states it plainly: a three-node control plane tolerates the failure of any single node, and losing more than one at the same time is a complete loss of service. Upgrading a control plane node takes its etcd member away for the reboot, so the cluster runs with no margin until it's back.",[11,1645,1646],{},"Talos' quorum guard stops the worst mistake, but I still gate each control plane node on a clean etcd status across all members before moving to the next:",[78,1648,1650],{"className":80,"code":1649,"language":82,"meta":83,"style":83},"talosctl etcd status --nodes 10.10.20.11,10.10.20.12,10.10.20.13\n",[18,1651,1652],{"__ignoreMap":83},[87,1653,1654,1656,1659,1662,1664],{"class":89,"line":90},[87,1655,75],{"class":93},[87,1657,1658],{"class":96}," etcd",[87,1660,1661],{"class":96}," status",[87,1663,104],{"class":103},[87,1665,1666],{"class":96}," 10.10.20.11,10.10.20.12,10.10.20.13\n",[78,1668,1671],{"className":1669,"code":1670,"language":113,"meta":83},[111],"NODE          MEMBER             DB SIZE   IN USE            LEADER             RAFT INDEX   RAFT TERM   RAFT APPLIED INDEX   LEARNER   ERRORS\n10.10.20.11   a49c021e76e707db   48 MB     21 MB (43.75%)    ecebb05b59a776f1   1982231      7           1982231              false\n10.10.20.12   ecebb05b59a776f1   47 MB     21 MB (44.68%)    ecebb05b59a776f1   1982231      7           1982231              false\n10.10.20.13   eb47fb33e59bf0e2   48 MB     21 MB (43.75%)    ecebb05b59a776f1   1982231      7           1982231              false\n",[18,1672,1670],{"__ignoreMap":83},[11,1674,1675,1676,1679,1680,1683,1684,1687],{},"All three members present, one agreed leader, matching raft indexes, nothing in ",[18,1677,1678],{},"ERRORS",". And before the first node is touched, I take an etcd snapshot. ",[18,1681,1682],{},"talosctl etcd snapshot"," produces a consistent snapshot from any healthy control plane node, and it's the file ",[18,1685,1686],{},"talosctl bootstrap --recover-from"," needs if everything goes wrong. A snapshot taken five minutes before the upgrade is worth far more than last night's.",[23,1689,1691],{"id":1690},"the-rolling-upgrade-as-a-script","The rolling upgrade as a script",[11,1693,1694,1695,1698],{},"This is the script, with the health gate between every node. ",[18,1696,1697],{},"talosctl health"," confirms etcd, the control plane components, node readiness and schedulability; the etcd status and alarm checks catch anything subtler. It stops at the first failure and leaves the remaining nodes on the old version, which is a perfectly supportable state for as long as it takes to investigate.",[78,1700,1702],{"className":80,"code":1701,"language":82,"meta":83,"style":83},"#!\u002Fusr\u002Fbin\u002Fenv bash\n# rolling-talos-upgrade.sh\n#\n# Upgrades Talos one node at a time: etcd snapshot first, then each control\n# plane node, then each worker, with a full health gate and an etcd check\n# between nodes. Stops at the first failure and leaves the rest untouched.\n#\n# Needs: talosctl matching the version the cluster runs now, an os:admin\n# talosconfig with the node IPs as endpoints, kubectl, and\n# upgrade_preflight.py next to this script.\nset -euo pipefail\n\nTARGET_VERSION=\"v1.14.0\"   # the latest patch of the next minor release; never skip a minor\nSCHEMATIC=\"\u003Cschematic-id>\" # from: talosctl get extensions --nodes \u003Cip>\nIMAGE=\"factory.talos.dev\u002Fmetal-installer\u002F${SCHEMATIC}:${TARGET_VERSION}\"\nCONTROL_PLANE=(10.10.20.11 10.10.20.12 10.10.20.13)\nWORKERS=(10.10.20.21 10.10.20.22)\nSNAPSHOT_DIR=\"\u002Fvar\u002Fbackups\u002Fetcd\"\n\nCP_CSV=\"$(IFS=,; echo \"${CONTROL_PLANE[*]}\")\"\nWORKER_CSV=\"$(IFS=,; echo \"${WORKERS[*]}\")\"\n\nhealth_gate() {\n    local args=(--nodes \"${CONTROL_PLANE[0]}\" --control-plane-nodes \"$CP_CSV\" --wait-timeout 10m)\n    if [[ ${#WORKERS[@]} -gt 0 ]]; then\n        args+=(--worker-nodes \"$WORKER_CSV\")\n    fi\n    talosctl health \"${args[@]}\"\n    talosctl etcd status --nodes \"$CP_CSV\"\n    if talosctl etcd alarm list --nodes \"$CP_CSV\" | grep -v '^NODE' | grep -q .; then\n        echo \"etcd has active alarms; stopping\" >&2\n        exit 1\n    fi\n}\n\nupgrade_node() {\n    local ip=\"$1\"\n    echo \"=== $(date -u +%FT%TZ) upgrading $ip to $TARGET_VERSION\"\n    python3 \"$(dirname \"$0\")\u002Fupgrade_preflight.py\"\n    talosctl upgrade --nodes \"$ip\" --image \"$IMAGE\" --wait --drain-timeout 10m\n    health_gate\n    talosctl version --nodes \"$ip\" | grep -A2 '^Server'\n}\n\necho \"=== pre-upgrade health and etcd snapshot\"\nhealth_gate\nmkdir -p \"$SNAPSHOT_DIR\"\ntalosctl etcd snapshot \"$SNAPSHOT_DIR\u002Fetcd-$(date -u +%Y%m%dT%H%M%SZ)-pre-${TARGET_VERSION}.snapshot\" \\\n    --nodes \"${CONTROL_PLANE[0]}\"\n\nfor ip in \"${CONTROL_PLANE[@]}\"; do\n    upgrade_node \"$ip\"\ndone\n\nfor ip in \"${WORKERS[@]}\"; do\n    upgrade_node \"$ip\"\ndone\n\necho \"=== all nodes on $TARGET_VERSION\"\nkubectl get nodes -o wide\n",[18,1703,1704,1709,1714,1719,1724,1729,1734,1738,1743,1748,1753,1763,1767,1780,1793,1813,1833,1850,1860,1864,1898,1925,1929,1937,1978,2008,2029,2034,2054,2071,2121,2132,2139,2143,2148,2152,2159,2175,2203,2222,2256,2261,2286,2290,2294,2301,2306,2321,2352,2364,2368,2393,2404,2409,2413,2435,2445,2449,2453,2464],{"__ignoreMap":83},[87,1705,1706],{"class":89,"line":90},[87,1707,1708],{"class":503},"#!\u002Fusr\u002Fbin\u002Fenv bash\n",[87,1710,1711],{"class":89,"line":161},[87,1712,1713],{"class":503},"# rolling-talos-upgrade.sh\n",[87,1715,1716],{"class":89,"line":172},[87,1717,1718],{"class":503},"#\n",[87,1720,1721],{"class":89,"line":181},[87,1722,1723],{"class":503},"# Upgrades Talos one node at a time: etcd snapshot first, then each control\n",[87,1725,1726],{"class":89,"line":192},[87,1727,1728],{"class":503},"# plane node, then each worker, with a full health gate and an etcd check\n",[87,1730,1731],{"class":89,"line":203},[87,1732,1733],{"class":503},"# between nodes. Stops at the first failure and leaves the rest untouched.\n",[87,1735,1736],{"class":89,"line":211},[87,1737,1718],{"class":503},[87,1739,1740],{"class":89,"line":222},[87,1741,1742],{"class":503},"# Needs: talosctl matching the version the cluster runs now, an os:admin\n",[87,1744,1745],{"class":89,"line":230},[87,1746,1747],{"class":503},"# talosconfig with the node IPs as endpoints, kubectl, and\n",[87,1749,1750],{"class":89,"line":238},[87,1751,1752],{"class":503},"# upgrade_preflight.py next to this script.\n",[87,1754,1755,1757,1760],{"class":89,"line":248},[87,1756,1290],{"class":103},[87,1758,1759],{"class":103}," -euo",[87,1761,1762],{"class":96}," pipefail\n",[87,1764,1765],{"class":89,"line":254},[87,1766,520],{"emptyLinePlaceholder":519},[87,1768,1769,1772,1774,1777],{"class":89,"line":264},[87,1770,1771],{"class":154},"TARGET_VERSION",[87,1773,644],{"class":577},[87,1775,1776],{"class":96},"\"v1.14.0\"",[87,1778,1779],{"class":503},"   # the latest patch of the next minor release; never skip a minor\n",[87,1781,1782,1785,1787,1790],{"class":89,"line":274},[87,1783,1784],{"class":154},"SCHEMATIC",[87,1786,644],{"class":577},[87,1788,1789],{"class":96},"\"\u003Cschematic-id>\"",[87,1791,1792],{"class":503}," # from: talosctl get extensions --nodes \u003Cip>\n",[87,1794,1795,1798,1800,1803,1805,1808,1810],{"class":89,"line":281},[87,1796,1797],{"class":154},"IMAGE",[87,1799,644],{"class":577},[87,1801,1802],{"class":96},"\"factory.talos.dev\u002Fmetal-installer\u002F${",[87,1804,1784],{"class":154},[87,1806,1807],{"class":96},"}:${",[87,1809,1771],{"class":154},[87,1811,1812],{"class":96},"}\"\n",[87,1814,1815,1818,1820,1822,1825,1828,1831],{"class":89,"line":290},[87,1816,1817],{"class":154},"CONTROL_PLANE",[87,1819,644],{"class":577},[87,1821,1483],{"class":154},[87,1823,1824],{"class":103},"10.10.20.11",[87,1826,1827],{"class":103}," 10.10.20.12",[87,1829,1830],{"class":103}," 10.10.20.13",[87,1832,659],{"class":154},[87,1834,1835,1838,1840,1842,1845,1848],{"class":89,"line":299},[87,1836,1837],{"class":154},"WORKERS",[87,1839,644],{"class":577},[87,1841,1483],{"class":154},[87,1843,1844],{"class":103},"10.10.20.21",[87,1846,1847],{"class":103}," 10.10.20.22",[87,1849,659],{"class":154},[87,1851,1852,1855,1857],{"class":89,"line":306},[87,1853,1854],{"class":154},"SNAPSHOT_DIR",[87,1856,644],{"class":577},[87,1858,1859],{"class":96},"\"\u002Fvar\u002Fbackups\u002Fetcd\"\n",[87,1861,1862],{"class":89,"line":317},[87,1863,520],{"emptyLinePlaceholder":519},[87,1865,1866,1869,1871,1874,1877,1879,1882,1885,1888,1890,1892,1895],{"class":89,"line":324},[87,1867,1868],{"class":154},"CP_CSV",[87,1870,644],{"class":577},[87,1872,1873],{"class":96},"\"$(",[87,1875,1876],{"class":154},"IFS",[87,1878,644],{"class":577},[87,1880,1881],{"class":96},",; ",[87,1883,1884],{"class":103},"echo",[87,1886,1887],{"class":96}," \"${",[87,1889,1817],{"class":154},[87,1891,716],{"class":96},[87,1893,1894],{"class":577},"*",[87,1896,1897],{"class":96},"]}\")\"\n",[87,1899,1900,1903,1905,1907,1909,1911,1913,1915,1917,1919,1921,1923],{"class":89,"line":331},[87,1901,1902],{"class":154},"WORKER_CSV",[87,1904,644],{"class":577},[87,1906,1873],{"class":96},[87,1908,1876],{"class":154},[87,1910,644],{"class":577},[87,1912,1881],{"class":96},[87,1914,1884],{"class":103},[87,1916,1887],{"class":96},[87,1918,1837],{"class":154},[87,1920,716],{"class":96},[87,1922,1894],{"class":577},[87,1924,1897],{"class":96},[87,1926,1927],{"class":89,"line":340},[87,1928,520],{"emptyLinePlaceholder":519},[87,1930,1931,1934],{"class":89,"line":348},[87,1932,1933],{"class":93},"health_gate",[87,1935,1936],{"class":154},"() {\n",[87,1938,1939,1942,1945,1947,1949,1952,1954,1956,1959,1962,1965,1968,1970,1973,1976],{"class":89,"line":356},[87,1940,1941],{"class":577},"    local",[87,1943,1944],{"class":154}," args",[87,1946,644],{"class":577},[87,1948,1483],{"class":154},[87,1950,1951],{"class":96},"--nodes",[87,1953,1887],{"class":96},[87,1955,1817],{"class":154},[87,1957,1958],{"class":96},"[0]}\"",[87,1960,1961],{"class":96}," --control-plane-nodes",[87,1963,1964],{"class":96}," \"",[87,1966,1967],{"class":154},"$CP_CSV",[87,1969,1228],{"class":96},[87,1971,1972],{"class":96}," --wait-timeout",[87,1974,1975],{"class":96}," 10m",[87,1977,659],{"class":154},[87,1979,1980,1982,1985,1988,1991,1994,1997,2000,2002,2005],{"class":89,"line":364},[87,1981,1506],{"class":577},[87,1983,1984],{"class":154}," [[ ${",[87,1986,1987],{"class":577},"#",[87,1989,1990],{"class":154},"WORKERS[",[87,1992,1993],{"class":577},"@",[87,1995,1996],{"class":154},"]} ",[87,1998,1999],{"class":577},"-gt",[87,2001,994],{"class":103},[87,2003,2004],{"class":154}," ]]; ",[87,2006,2007],{"class":577},"then\n",[87,2009,2010,2013,2015,2017,2020,2022,2025,2027],{"class":89,"line":374},[87,2011,2012],{"class":154},"        args",[87,2014,1113],{"class":577},[87,2016,1483],{"class":154},[87,2018,2019],{"class":96},"--worker-nodes",[87,2021,1964],{"class":96},[87,2023,2024],{"class":154},"$WORKER_CSV",[87,2026,1228],{"class":96},[87,2028,659],{"class":154},[87,2030,2031],{"class":89,"line":382},[87,2032,2033],{"class":577},"    fi\n",[87,2035,2036,2039,2042,2044,2047,2049,2051],{"class":89,"line":390},[87,2037,2038],{"class":93},"    talosctl",[87,2040,2041],{"class":96}," health",[87,2043,1887],{"class":96},[87,2045,2046],{"class":154},"args",[87,2048,716],{"class":96},[87,2050,1993],{"class":577},[87,2052,2053],{"class":96},"]}\"\n",[87,2055,2056,2058,2060,2062,2064,2066,2068],{"class":89,"line":403},[87,2057,2038],{"class":93},[87,2059,1658],{"class":96},[87,2061,1661],{"class":96},[87,2063,104],{"class":103},[87,2065,1964],{"class":96},[87,2067,1967],{"class":154},[87,2069,2070],{"class":96},"\"\n",[87,2072,2073,2075,2078,2080,2083,2086,2088,2090,2092,2094,2097,2100,2103,2106,2108,2110,2113,2116,2119],{"class":89,"line":414},[87,2074,1506],{"class":577},[87,2076,2077],{"class":93}," talosctl",[87,2079,1658],{"class":96},[87,2081,2082],{"class":96}," alarm",[87,2084,2085],{"class":96}," list",[87,2087,104],{"class":103},[87,2089,1964],{"class":96},[87,2091,1967],{"class":154},[87,2093,1228],{"class":96},[87,2095,2096],{"class":577}," |",[87,2098,2099],{"class":93}," grep",[87,2101,2102],{"class":103}," -v",[87,2104,2105],{"class":96}," '^NODE'",[87,2107,2096],{"class":577},[87,2109,2099],{"class":93},[87,2111,2112],{"class":103}," -q",[87,2114,2115],{"class":96}," .",[87,2117,2118],{"class":154},"; ",[87,2120,2007],{"class":577},[87,2122,2123,2126,2129],{"class":89,"line":425},[87,2124,2125],{"class":103},"        echo",[87,2127,2128],{"class":96}," \"etcd has active alarms; stopping\"",[87,2130,2131],{"class":577}," >&2\n",[87,2133,2134,2137],{"class":89,"line":433},[87,2135,2136],{"class":103},"        exit",[87,2138,1183],{"class":103},[87,2140,2141],{"class":89,"line":441},[87,2142,2033],{"class":577},[87,2144,2145],{"class":89,"line":451},[87,2146,2147],{"class":154},"}\n",[87,2149,2150],{"class":89,"line":459},[87,2151,520],{"emptyLinePlaceholder":519},[87,2153,2154,2157],{"class":89,"line":472},[87,2155,2156],{"class":93},"upgrade_node",[87,2158,1936],{"class":154},[87,2160,2161,2163,2166,2168,2170,2173],{"class":89,"line":864},[87,2162,1941],{"class":577},[87,2164,2165],{"class":154}," ip",[87,2167,644],{"class":577},[87,2169,1228],{"class":96},[87,2171,2172],{"class":103},"$1",[87,2174,2070],{"class":96},[87,2176,2177,2180,2183,2186,2189,2192,2195,2198,2201],{"class":89,"line":893},[87,2178,2179],{"class":103},"    echo",[87,2181,2182],{"class":96}," \"=== $(",[87,2184,2185],{"class":93},"date",[87,2187,2188],{"class":103}," -u",[87,2190,2191],{"class":96}," +%FT%TZ) upgrading ",[87,2193,2194],{"class":154},"$ip",[87,2196,2197],{"class":96}," to ",[87,2199,2200],{"class":154},"$TARGET_VERSION",[87,2202,2070],{"class":96},[87,2204,2205,2208,2211,2214,2216,2219],{"class":89,"line":919},[87,2206,2207],{"class":93},"    python3",[87,2209,2210],{"class":96}," \"$(",[87,2212,2213],{"class":93},"dirname",[87,2215,1964],{"class":96},[87,2217,2218],{"class":103},"$0",[87,2220,2221],{"class":96},"\")\u002Fupgrade_preflight.py\"\n",[87,2223,2224,2226,2229,2231,2233,2235,2237,2240,2242,2245,2247,2250,2253],{"class":89,"line":927},[87,2225,2038],{"class":93},[87,2227,2228],{"class":96}," upgrade",[87,2230,104],{"class":103},[87,2232,1964],{"class":96},[87,2234,2194],{"class":154},[87,2236,1228],{"class":96},[87,2238,2239],{"class":103}," --image",[87,2241,1964],{"class":96},[87,2243,2244],{"class":154},"$IMAGE",[87,2246,1228],{"class":96},[87,2248,2249],{"class":103}," --wait",[87,2251,2252],{"class":103}," --drain-timeout",[87,2254,2255],{"class":96}," 10m\n",[87,2257,2258],{"class":89,"line":947},[87,2259,2260],{"class":93},"    health_gate\n",[87,2262,2263,2265,2268,2270,2272,2274,2276,2278,2280,2283],{"class":89,"line":952},[87,2264,2038],{"class":93},[87,2266,2267],{"class":96}," version",[87,2269,104],{"class":103},[87,2271,1964],{"class":96},[87,2273,2194],{"class":154},[87,2275,1228],{"class":96},[87,2277,2096],{"class":577},[87,2279,2099],{"class":93},[87,2281,2282],{"class":103}," -A2",[87,2284,2285],{"class":96}," '^Server'\n",[87,2287,2288],{"class":89,"line":965},[87,2289,2147],{"class":154},[87,2291,2292],{"class":89,"line":978},[87,2293,520],{"emptyLinePlaceholder":519},[87,2295,2296,2298],{"class":89,"line":984},[87,2297,1884],{"class":103},[87,2299,2300],{"class":96}," \"=== pre-upgrade health and etcd snapshot\"\n",[87,2302,2303],{"class":89,"line":1008},[87,2304,2305],{"class":93},"health_gate\n",[87,2307,2308,2311,2314,2316,2319],{"class":89,"line":1014},[87,2309,2310],{"class":93},"mkdir",[87,2312,2313],{"class":103}," -p",[87,2315,1964],{"class":96},[87,2317,2318],{"class":154},"$SNAPSHOT_DIR",[87,2320,2070],{"class":96},[87,2322,2323,2325,2327,2330,2332,2334,2337,2339,2341,2344,2346,2349],{"class":89,"line":1043},[87,2324,75],{"class":93},[87,2326,1658],{"class":96},[87,2328,2329],{"class":96}," snapshot",[87,2331,1964],{"class":96},[87,2333,2318],{"class":154},[87,2335,2336],{"class":96},"\u002Fetcd-$(",[87,2338,2185],{"class":93},[87,2340,2188],{"class":103},[87,2342,2343],{"class":96}," +%Y%m%dT%H%M%SZ)-pre-${",[87,2345,1771],{"class":154},[87,2347,2348],{"class":96},"}.snapshot\"",[87,2350,2351],{"class":103}," \\\n",[87,2353,2354,2357,2359,2361],{"class":89,"line":1070},[87,2355,2356],{"class":103},"    --nodes",[87,2358,1887],{"class":96},[87,2360,1817],{"class":154},[87,2362,2363],{"class":96},"[0]}\"\n",[87,2365,2366],{"class":89,"line":1076},[87,2367,520],{"emptyLinePlaceholder":519},[87,2369,2370,2372,2375,2377,2379,2381,2383,2385,2388,2390],{"class":89,"line":1081},[87,2371,828],{"class":577},[87,2373,2374],{"class":154}," ip ",[87,2376,809],{"class":577},[87,2378,1887],{"class":96},[87,2380,1817],{"class":154},[87,2382,716],{"class":96},[87,2384,1993],{"class":577},[87,2386,2387],{"class":96},"]}\"",[87,2389,2118],{"class":154},[87,2391,2392],{"class":577},"do\n",[87,2394,2395,2398,2400,2402],{"class":89,"line":1108},[87,2396,2397],{"class":93},"    upgrade_node",[87,2399,1964],{"class":96},[87,2401,2194],{"class":154},[87,2403,2070],{"class":96},[87,2405,2406],{"class":89,"line":1134},[87,2407,2408],{"class":577},"done\n",[87,2410,2411],{"class":89,"line":1147},[87,2412,520],{"emptyLinePlaceholder":519},[87,2414,2415,2417,2419,2421,2423,2425,2427,2429,2431,2433],{"class":89,"line":1158},[87,2416,828],{"class":577},[87,2418,2374],{"class":154},[87,2420,809],{"class":577},[87,2422,1887],{"class":96},[87,2424,1837],{"class":154},[87,2426,716],{"class":96},[87,2428,1993],{"class":577},[87,2430,2387],{"class":96},[87,2432,2118],{"class":154},[87,2434,2392],{"class":577},[87,2436,2437,2439,2441,2443],{"class":89,"line":1186},[87,2438,2397],{"class":93},[87,2440,1964],{"class":96},[87,2442,2194],{"class":154},[87,2444,2070],{"class":96},[87,2446,2447],{"class":89,"line":1221},[87,2448,2408],{"class":577},[87,2450,2451],{"class":89,"line":1268},[87,2452,520],{"emptyLinePlaceholder":519},[87,2454,2455,2457,2460,2462],{"class":89,"line":1273},[87,2456,1884],{"class":103},[87,2458,2459],{"class":96}," \"=== all nodes on ",[87,2461,2200],{"class":154},[87,2463,2070],{"class":96},[87,2465,2466,2469,2471,2474,2477],{"class":89,"line":1279},[87,2467,2468],{"class":93},"kubectl",[87,2470,97],{"class":96},[87,2472,2473],{"class":96}," nodes",[87,2475,2476],{"class":103}," -o",[87,2478,2479],{"class":96}," wide\n",[11,2481,2482,2483,2485],{},"Upgrade to the latest patch of each intermediate minor release rather than jumping: Talos only tests config migrations between adjacent minors. Going from 1.12 to 1.14 means two passes of this script, 1.12 to the latest 1.13 patch and then 1.13 to 1.14. Sidero also recommends running the ",[18,2484,75],{}," version that matches the version the cluster currently runs.",[23,2487,2489],{"id":2488},"sequence-the-os-and-kubernetes-upgrades-separately","Sequence the OS and Kubernetes upgrades separately",[11,2491,2492,2493,2495,2496,2498],{},"It's tempting to bump the Talos version and the Kubernetes version in the same pass, but I treat them as two rolling operations with a validation gap in between. Talos made that the default years ago: since 1.0, a Talos OS upgrade doesn't upgrade Kubernetes. ",[18,2494,29],{}," handles the OS, and ",[18,2497,134],{}," handles the control plane components, kube-proxy, the kubelet on every node, and the bootstrap manifests. When both changes land in the same pass and something breaks, the first question is always \"which of the two things I changed caused this,\" and separating them removes that question entirely.",[11,2500,2501],{},"The Kubernetes step has a dry run that I always read first. It lists resources using APIs that are being deprecated in the target version and shows every component it would update:",[78,2503,2505],{"className":80,"code":2504,"language":82,"meta":83,"style":83},"talosctl --nodes 10.10.20.11 upgrade-k8s --to 1.37.0 --dry-run\ntalosctl --nodes 10.10.20.11 upgrade-k8s --to 1.37.0\n",[18,2506,2507,2528],{"__ignoreMap":83},[87,2508,2509,2511,2513,2516,2519,2522,2525],{"class":89,"line":90},[87,2510,75],{"class":93},[87,2512,104],{"class":103},[87,2514,2515],{"class":103}," 10.10.20.11",[87,2517,2518],{"class":96}," upgrade-k8s",[87,2520,2521],{"class":103}," --to",[87,2523,2524],{"class":103}," 1.37.0",[87,2526,2527],{"class":103}," --dry-run\n",[87,2529,2530,2532,2534,2536,2538,2540],{"class":89,"line":161},[87,2531,75],{"class":93},[87,2533,104],{"class":103},[87,2535,2515],{"class":103},[87,2537,2518],{"class":96},[87,2539,2521],{"class":103},[87,2541,2542],{"class":103}," 1.37.0\n",[11,2544,2545,2547,2548,2551],{},[18,2546,1951],{}," names the control plane node that receives the API call, but every node in the cluster gets upgraded. The command pre-pulls images, patches each control plane node's config with the new component versions, updates kube-proxy, then upgrades and verifies the kubelet node by node. If it fails partway, you can rerun it and it continues from where it stopped. Go one Kubernetes minor at a time: the version skew policy doesn't allow ",[18,2549,2550],{},"kube-apiserver"," to skip minor versions. Check the target against the Talos support matrix, too; Talos 1.14 supports Kubernetes 1.33 through 1.37.",[11,2553,2554,2555,2558,2559,2562,2563,2566],{},"One follow-up belongs in the change ticket. ",[18,2556,2557],{},"upgrade-k8s"," rewrites component versions in each node's live config, so any full ",[18,2560,2561],{},"controlplane.yaml"," you kept in Git is now stale and would downgrade components if re-applied. That's the drift Sidero's reproducible-configuration guide is written around: keep ",[18,2564,2565],{},"secrets.yaml"," and patches, and regenerate full configs with the Kubernetes version the cluster now runs.",[23,2568,2570],{"id":2569},"what-actually-made-this-boring","What actually made this boring",[11,2572,2573,2574,2577,2578,2581,2582,2585,2586,2590,2591,2594],{},"The upgrades that go well are the ones where I've already tested the exact upgrade path, not just \"some Talos upgrade\" but the specific current-version-to-target-version jump, on a non-production cluster running comparable workloads. Every release's upgrade guide has a \"before upgrade\" section, and it's not boilerplate: the 1.14 guide, for example, requires migrating the ",[18,2575,2576],{},"multipath-tools"," extension's configuration from an ",[18,2579,2580],{},"ExtensionServiceConfig"," document to an ",[18,2583,2584],{},"EtcFileConfig"," document ",[2587,2588,2589],"em",{},"before"," upgrading, or ",[18,2592,2593],{},"multipathd"," waits forever after the reboot. Finding that in a test cluster costs nothing. Finding it mid-rollout on a production control plane costs the calm that makes a no-downtime upgrade worth attempting in the first place. Once the workload placement is right and the upgrade path is proven, the rolling upgrade itself really is close to a non-event, which is the whole point.",[11,2596,2597,2598,2603],{},"The cron health check in ",[2599,2600,2602],"a",{"href":2601},"\u002F2026\u002F02\u002F04\u002Fpython-talos-health-checking-a-kubernetes-cluster-from-a-cron-job\u002F","Python: Talos – Health-Checking a Kubernetes Cluster from a Cron Job"," is what tells me, the morning after, that nothing was left cordoned or crash-looping.",[23,2605,2607],{"id":2606},"references","References",[45,2609,2610,2619,2630,2641,2654,2666,2673,2690],{},[48,2611,2612,2618],{},[2599,2613,2617],{"href":2614,"rel":2615},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fconfigure-your-talos-cluster\u002Flifecycle-management\u002Fupgrading-talos",[2616],"nofollow","Upgrading Talos Linux"," (sequence, A-B rollback, quorum guard, supported paths, Factory installer images)",[48,2620,2621,2626,2627,2629],{},[2599,2622,2625],{"href":2623,"rel":2624},"https:\u002F\u002Fdocs.siderolabs.com\u002Fkubernetes-guides\u002Fadvanced-guides\u002Fupgrading-kubernetes",[2616],"Upgrading Kubernetes"," (",[18,2628,2557],{},", dry run, phases, config drift)",[48,2631,2632,2637,2638,2640],{},[2599,2633,2636],{"href":2634,"rel":2635},"https:\u002F\u002Fgithub.com\u002Fsiderolabs\u002Ftalos\u002Fdiscussions\u002F9359",[2616],"Talos v1.8.0 release notes"," (installer never wipes the system disk; ",[18,2639,55],{}," always set)",[48,2642,2643,2648,2649],{},[2599,2644,2647],{"href":2645,"rel":2646},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fbuild-and-extend-talos\u002Fcluster-operations-and-maintenance\u002Fdisaster-recovery",[2616],"Disaster Recovery"," and ",[2599,2650,2653],{"href":2651,"rel":2652},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fbuild-and-extend-talos\u002Fcluster-operations-and-maintenance\u002Fetcd-maintenance",[2616],"etcd Maintenance",[48,2655,2656,2661,2662,2665],{},[2599,2657,2660],{"href":2658,"rel":2659},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Flearn-more\u002Fimage-factory",[2616],"Image Factory"," (schematic IDs, ",[18,2663,2664],{},"talosctl get extensions",")",[48,2667,2668],{},[2599,2669,2672],{"href":2670,"rel":2671},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fgetting-started\u002Fsupport-matrix",[2616],"Support Matrix",[48,2674,2675,670,2680,2648,2685],{},[2599,2676,2679],{"href":2677,"rel":2678},"https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Fconcepts\u002Fworkloads\u002Fpods\u002Fdisruptions\u002F",[2616],"Kubernetes: Disruptions",[2599,2681,2684],{"href":2682,"rel":2683},"https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Ftasks\u002Frun-application\u002Fconfigure-pdb\u002F",[2616],"Specifying a Disruption Budget",[2599,2686,2689],{"href":2687,"rel":2688},"https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Fconcepts\u002Fscheduling-eviction\u002Ftopology-spread-constraints\u002F",[2616],"Pod Topology Spread Constraints",[48,2691,2692],{},[2599,2693,2696],{"href":2694,"rel":2695},"https:\u002F\u002Fkubernetes.io\u002Freleases\u002Fversion-skew-policy\u002F",[2616],"Kubernetes Version Skew Policy",[2698,2699,2700],"style",{},"html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .s4JwU, html code.shiki .s4JwU{--shiki-default:#85E89D}html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .sAwPA, html code.shiki .sAwPA{--shiki-default:#6A737D}html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}html pre.shiki code .s9osk, html code.shiki .s9osk{--shiki-default:#FFAB70}",{"title":83,"searchDepth":161,"depth":161,"links":2702},[2703,2705,2706,2707,2708,2709,2710],{"id":25,"depth":161,"text":2704},"What talosctl upgrade does, and what it doesn't",{"id":124,"depth":161,"text":125},{"id":1639,"depth":161,"text":1640},{"id":1690,"depth":161,"text":1691},{"id":2488,"depth":161,"text":2489},{"id":2569,"depth":161,"text":2570},{"id":2606,"depth":161,"text":2607},"techcolumnist",[2713],"engineering","2026-08-26T14:00:00Z","Rolling Talos and Kubernetes upgrades through a cluster one node at a time: preflight checks, etcd snapshots, Image Factory installers, and scripts.","md",false,null,{},"\u002Fblog\u002F2026\u002F08\u002F26\u002Ftalos-linux-upgrading-a-cluster-without-a-maintenance-window",{"title":6,"description":2715},[2711],"blog\u002F2026\u002F08\u002F26\u002Ftalos-linux-upgrading-a-cluster-without-a-maintenance-window",[2725,2726],"talos","kubernetes","2026-09-09T01:25:30Z","\u002F2026\u002F08\u002F26\u002Ftalos-linux-upgrading-a-cluster-without-a-maintenance-window\u002F","58gCImjx_jLhPHiNnyOG-1ubaAdtghqz_LfKy0bsojE",{"title":2731,"description":2732,"date":2733,"url":2734,"categories":2735,"tags":2737,"image":2718,"readingTime":192,"canonical":2711,"sites":2741,"series":2718,"seriesOrder":2718},"Windows Management: A Discovery Inventory Nobody Maintains by Hand","Why a self-refreshing Windows inventory built from AD and CIM discovery outlasts a spreadsheet, with the sweep script, staleness report and schedule.","2026-09-02T14:00:00Z","\u002F2026\u002F09\u002F02\u002Fwindows-management-building-a-discovery-inventory-nobody-has-to-maintain-by-hand\u002F",[2713,2736],"strategy",[2738,2739,2740],"discovery","windows","active-directory",[2711],{"title":2743,"description":2744,"date":2745,"url":2746,"categories":2747,"tags":2748,"image":2718,"readingTime":211,"canonical":2711,"sites":2752,"series":2718,"seriesOrder":2718},"Data Warehouse: Reusable SQL Models for IT Operations Metrics","Layered, version-controlled SQL models for MTTR, ticket aging, SLA compliance, and patch compliance, with plain-SQL tests and the dbt equivalent.","2026-08-19T14:00:00Z","\u002F2026\u002F08\u002F19\u002Fdata-warehouse-reusable-sql-models-for-it-operations-metrics\u002F",[2713],[2749,2750,2751],"data-warehouse","reporting","sql",[2711],[2754,2763,2771],{"title":2755,"description":2756,"date":2757,"url":2758,"categories":2759,"tags":2760,"image":2718,"readingTime":192,"canonical":2711,"sites":2762,"series":2718,"seriesOrder":2718},"Hyper-V to Talos: Rethinking the Virtualization Stack","Why moving workloads off Hyper-V VMs onto a Talos Kubernetes cluster changed the whole stack, with an inventory script and a VM-to-manifest example.","2026-09-16T14:00:00Z","\u002F2026\u002F09\u002F16\u002Fhyper-v-to-talos-rethinking-the-virtualization-stack\u002F",[2713,2736],[2761,2725,2726],"hyper-v",[2711],{"title":2764,"description":2765,"date":2766,"url":2767,"categories":2768,"tags":2769,"image":2718,"readingTime":211,"canonical":2711,"sites":2770,"series":2718,"seriesOrder":2718},"Kubernetes: Running Talos Alongside a Legacy Hyper-V Estate","Introducing a Talos Kubernetes cluster into an existing Hyper-V estate: VM build script, Image Factory schematic, VLAN, VIP, firewall and storage boundaries.","2026-07-08T14:00:00Z","\u002F2026\u002F07\u002F08\u002Fkubernetes-running-talos-alongside-a-legacy-hyper-v-estate\u002F",[2713],[2725,2726,2761],[2711],{"title":2772,"description":2773,"date":2774,"url":2775,"categories":2776,"tags":2777,"image":2718,"readingTime":203,"canonical":2711,"sites":2778,"series":2718,"seriesOrder":2718},"Talos Linux: What Immutable Infrastructure Buys You in Production","Where an immutable, API-managed OS pays off under Kubernetes and where it bites: drift, upgrades, try-mode config changes, and debugging without SSH.","2026-03-25T14:00:00Z","\u002F2026\u002F03\u002F25\u002Ftalos-linux-what-immutable-infrastructure-actually-buys-you-in-production\u002F",[2713,2736],[2725,2726],[2711],{"doc":2718,"posts":2780},[],1790052514107]