[{"data":1,"prerenderedAt":2262},["ShallowReactive",2],{"post:\u002F2026\u002F03\u002F25\u002Ftalos-linux-what-immutable-infrastructure-actually-buys-you-in-production\u002F":3},{"post":4,"newer":2215,"older":2227,"related":2234,"series":2260},{"id":5,"title":6,"body":7,"canonical":2195,"categories":2196,"date":2199,"description":2200,"extension":2201,"featured":2202,"hero":2203,"image":2203,"meta":2204,"navigation":97,"path":2205,"readingTime":107,"seo":2206,"series":2203,"seriesOrder":2203,"sites":2207,"source":2203,"stem":2208,"tags":2209,"updated":2212,"url":2213,"__hash__":2214},"blog\u002Fblog\u002F2026\u002F03\u002F25\u002Ftalos-linux-what-immutable-infrastructure-actually-buys-you-in-production.md","Talos Linux: What Immutable Infrastructure Buys You in Production",{"type":8,"value":9,"toc":2187},"minimark",[10,14,17,22,34,41,61,1475,1478,1571,1578,1593,1602,1606,1617,1624,1709,1712,1759,1773,1780,1784,1791,2058,2080,2084,2087,2091,2097,2101,2183],[11,12,13],"p",{},"I've spent enough years patching Ubuntu and RHEL nodes under Kubernetes clusters to be skeptical of any pitch that starts with \"just don't SSH into it.\" Talos Linux makes that pitch literally: there's no shell, no SSH daemon, no package manager, not even busybox, and the entire OS is managed through a gRPC API and a declarative machine configuration. After running it under a few production clusters, the skepticism was partly warranted, but the parts of the pitch that hold up are worth more than I expected.",[11,15,16],{},"Everything below is written against Talos 1.14.",[18,19,21],"h2",{"id":20},"what-immutable-actually-removes","What \"immutable\" actually removes",[11,23,24,25,29,30,33],{},"Talos' philosophy page is specific about what immutability means. The root filesystem is a read-only SquashFS image, signed and delivered as a single versioned file, and Talos runs from that image even when installed to disk. There are a few controlled writable locations, and the main writable partition is deliberately called ",[26,27,28],"code",{},"EPHEMERAL"," to remind everyone not to keep anything unique there. PID 1 is Talos' own ",[26,31,32],{},"machined",", not systemd. User space is a ground-up rewrite in Go, not a trimmed-down distribution.",[11,35,36,37,40],{},"The obvious sell is security surface: no shell means no shell-based lateral movement, and no accumulated cruft from an admin who SSH'd in at 2am and left a fix in place that never made it back into config management. That's real, but the bigger day-to-day win is different. There's no longer a category of incident where the node's actual state has quietly drifted from what your infrastructure-as-code says it should be. On a normal distro, drift is a slow leak: a manual ",[26,38,39],{},"apt install"," here, a kernel parameter tweaked to fix an incident there, none of it ever reverted. On Talos the only way to change a node is to change its machine config through the API, and the running config is a resource you can read back.",[11,42,43,44,48,49,52,53,56,57,60],{},"That last property is what makes drift ",[45,46,47],"em",{},"checkable",", not just unlikely. The running config comes back with ",[26,50,51],{},"talosctl get machineconfig v1alpha1 -o jsonpath='{.spec}'",". If you manage clusters the way Sidero Labs now recommends (a ",[26,54,55],{},"secrets.yaml"," bundle plus small patch files, with full configs regenerated on demand rather than committed), you can regenerate what each node ",[45,58,59],{},"should"," be running and compare. This is the script I run weekly and before every upgrade:",[62,63,68],"pre",{"className":64,"code":65,"language":66,"meta":67,"style":67},"language-python shiki shiki-themes github-dark","#!\u002Fusr\u002Fbin\u002Fenv python3\n\"\"\"\ntalos_drift_report.py\n\nCompares the machine configuration running on each Talos node with the\nconfiguration regenerated from secrets.yaml + patches (the declared state).\nBoth sides are parsed and normalized (documents keyed by kind\u002Fname, keys\nsorted) so comment and ordering noise doesn't show up as drift.\n\nExit code 0 when every node matches, 1 when any node has drifted.\n\nRequires: talosctl on PATH, PyYAML (pip install pyyaml).\nOutput can contain machine config values: keep the report private.\n\"\"\"\n\nimport argparse\nimport difflib\nimport subprocess\nimport sys\nimport tempfile\nfrom pathlib import Path\n\nimport yaml\n\n\ndef normalize(text):\n    \"\"\"Return the config as sorted YAML, one entry per (kind, name) document.\"\"\"\n    docs = {}\n    for doc in yaml.safe_load_all(text):\n        if not doc:\n            continue\n        key = f\"{doc.get('kind', 'v1alpha1')}\u002F{doc.get('name', '')}\"\n        docs[key] = doc\n    return yaml.safe_dump(docs, sort_keys=True, default_flow_style=False).splitlines()\n\n\ndef declared_config(args, node_name):\n    \"\"\"Regenerate the node's controlplane config the same way it was created.\"\"\"\n    with tempfile.TemporaryDirectory() as tmp:\n        out = Path(tmp) \u002F f\"{node_name}.yaml\"\n        cmd = [\n            \"talosctl\", \"gen\", \"config\", args.cluster_name, args.endpoint,\n            \"--with-secrets\", args.secrets,\n            \"--kubernetes-version\", args.kubernetes_version,\n            \"--talos-version\", args.talos_contract,\n            \"--with-docs=false\", \"--with-examples=false\",\n            \"--output-types\", \"controlplane\",\n            \"--output\", str(out),\n        ]\n        for patch in (\"common.yaml\", \"controlplane.yaml\", f\"nodes\u002F{node_name}.yaml\"):\n            cmd += [\"--config-patch\", f\"@{Path(args.patch_dir) \u002F patch}\"]\n        subprocess.run(cmd, check=True, capture_output=True, text=True)\n        return out.read_text()\n\n\ndef live_config(ip):\n    \"\"\"Fetch the running config: the .spec of the MachineConfig resource.\"\"\"\n    result = subprocess.run(\n        [\"talosctl\", \"--nodes\", ip, \"get\", \"machineconfig\", \"v1alpha1\", \"-o\", \"jsonpath={.spec}\"],\n        check=True, capture_output=True, text=True,\n    )\n    return result.stdout\n\n\ndef main():\n    parser = argparse.ArgumentParser(description=\"Report drift between declared and running Talos configs.\")\n    parser.add_argument(\"--cluster-name\", required=True)\n    parser.add_argument(\"--endpoint\", required=True, help=\"Kubernetes endpoint, e.g. https:\u002F\u002F10.10.20.10:6443\")\n    parser.add_argument(\"--secrets\", default=\"secrets.yaml\")\n    parser.add_argument(\"--patch-dir\", default=\"patches\")\n    parser.add_argument(\"--kubernetes-version\", required=True, help=\"Version the cluster runs now\")\n    parser.add_argument(\"--talos-contract\", required=True, help=\"Talos version contract used at creation, e.g. v1.14\")\n    parser.add_argument(\"--node\", action=\"append\", required=True, metavar=\"NAME=IP\",\n                        help=\"Node name (matching patches\u002Fnodes\u002FNAME.yaml) and IP; repeatable\")\n    args = parser.parse_args()\n\n    drifted = 0\n    for entry in args.node:\n        name, ip = entry.split(\"=\", 1)\n        try:\n            declared = normalize(declared_config(args, name))\n            live = normalize(live_config(ip))\n        except subprocess.CalledProcessError as exc:\n            print(f\"[{name}] ERROR: {' '.join(exc.cmd[:3])} failed: {exc.stderr.strip()}\")\n            drifted += 1\n            continue\n\n        diff = list(difflib.unified_diff(declared, live, fromfile=f\"{name} declared\", tofile=f\"{name} running\", lineterm=\"\"))\n        if diff:\n            drifted += 1\n            print(f\"[{name}] DRIFT ({ip}):\")\n            print(\"\\n\".join(diff))\n        else:\n            print(f\"[{name}] in sync ({ip})\")\n\n    sys.exit(1 if drifted else 0)\n\n\nif __name__ == \"__main__\":\n    main()\n","python","",[26,69,70,79,86,92,99,105,111,117,123,128,134,139,145,151,156,161,172,180,188,196,204,218,223,231,236,241,254,260,272,287,299,305,363,374,405,410,415,426,432,447,474,485,504,513,522,531,545,558,572,578,620,659,693,702,707,712,723,729,740,788,816,822,830,835,840,851,872,892,920,940,959,986,1013,1051,1064,1075,1080,1091,1104,1125,1134,1145,1156,1170,1224,1235,1240,1245,1310,1318,1327,1359,1376,1384,1415,1420,1442,1447,1452,1469],{"__ignoreMap":67},[71,72,75],"span",{"class":73,"line":74},"line",1,[71,76,78],{"class":77},"sAwPA","#!\u002Fusr\u002Fbin\u002Fenv python3\n",[71,80,82],{"class":73,"line":81},2,[71,83,85],{"class":84},"sU2Wk","\"\"\"\n",[71,87,89],{"class":73,"line":88},3,[71,90,91],{"class":84},"talos_drift_report.py\n",[71,93,95],{"class":73,"line":94},4,[71,96,98],{"emptyLinePlaceholder":97},true,"\n",[71,100,102],{"class":73,"line":101},5,[71,103,104],{"class":84},"Compares the machine configuration running on each Talos node with the\n",[71,106,108],{"class":73,"line":107},6,[71,109,110],{"class":84},"configuration regenerated from secrets.yaml + patches (the declared state).\n",[71,112,114],{"class":73,"line":113},7,[71,115,116],{"class":84},"Both sides are parsed and normalized (documents keyed by kind\u002Fname, keys\n",[71,118,120],{"class":73,"line":119},8,[71,121,122],{"class":84},"sorted) so comment and ordering noise doesn't show up as drift.\n",[71,124,126],{"class":73,"line":125},9,[71,127,98],{"emptyLinePlaceholder":97},[71,129,131],{"class":73,"line":130},10,[71,132,133],{"class":84},"Exit code 0 when every node matches, 1 when any node has drifted.\n",[71,135,137],{"class":73,"line":136},11,[71,138,98],{"emptyLinePlaceholder":97},[71,140,142],{"class":73,"line":141},12,[71,143,144],{"class":84},"Requires: talosctl on PATH, PyYAML (pip install pyyaml).\n",[71,146,148],{"class":73,"line":147},13,[71,149,150],{"class":84},"Output can contain machine config values: keep the report private.\n",[71,152,154],{"class":73,"line":153},14,[71,155,85],{"class":84},[71,157,159],{"class":73,"line":158},15,[71,160,98],{"emptyLinePlaceholder":97},[71,162,164,168],{"class":73,"line":163},16,[71,165,167],{"class":166},"snl16","import",[71,169,171],{"class":170},"s95oV"," argparse\n",[71,173,175,177],{"class":73,"line":174},17,[71,176,167],{"class":166},[71,178,179],{"class":170}," difflib\n",[71,181,183,185],{"class":73,"line":182},18,[71,184,167],{"class":166},[71,186,187],{"class":170}," subprocess\n",[71,189,191,193],{"class":73,"line":190},19,[71,192,167],{"class":166},[71,194,195],{"class":170}," sys\n",[71,197,199,201],{"class":73,"line":198},20,[71,200,167],{"class":166},[71,202,203],{"class":170}," tempfile\n",[71,205,207,210,213,215],{"class":73,"line":206},21,[71,208,209],{"class":166},"from",[71,211,212],{"class":170}," pathlib ",[71,214,167],{"class":166},[71,216,217],{"class":170}," Path\n",[71,219,221],{"class":73,"line":220},22,[71,222,98],{"emptyLinePlaceholder":97},[71,224,226,228],{"class":73,"line":225},23,[71,227,167],{"class":166},[71,229,230],{"class":170}," yaml\n",[71,232,234],{"class":73,"line":233},24,[71,235,98],{"emptyLinePlaceholder":97},[71,237,239],{"class":73,"line":238},25,[71,240,98],{"emptyLinePlaceholder":97},[71,242,244,247,251],{"class":73,"line":243},26,[71,245,246],{"class":166},"def",[71,248,250],{"class":249},"svObZ"," normalize",[71,252,253],{"class":170},"(text):\n",[71,255,257],{"class":73,"line":256},27,[71,258,259],{"class":84},"    \"\"\"Return the config as sorted YAML, one entry per (kind, name) document.\"\"\"\n",[71,261,263,266,269],{"class":73,"line":262},28,[71,264,265],{"class":170},"    docs ",[71,267,268],{"class":166},"=",[71,270,271],{"class":170}," {}\n",[71,273,275,278,281,284],{"class":73,"line":274},29,[71,276,277],{"class":166},"    for",[71,279,280],{"class":170}," doc ",[71,282,283],{"class":166},"in",[71,285,286],{"class":170}," yaml.safe_load_all(text):\n",[71,288,290,293,296],{"class":73,"line":289},30,[71,291,292],{"class":166},"        if",[71,294,295],{"class":166}," not",[71,297,298],{"class":170}," doc:\n",[71,300,302],{"class":73,"line":301},31,[71,303,304],{"class":166},"            continue\n",[71,306,308,311,313,316,319,323,326,329,332,335,338,341,344,346,348,351,353,356,358,360],{"class":73,"line":307},32,[71,309,310],{"class":170},"        key ",[71,312,268],{"class":166},[71,314,315],{"class":166}," f",[71,317,318],{"class":84},"\"",[71,320,322],{"class":321},"sDLfK","{",[71,324,325],{"class":170},"doc.get(",[71,327,328],{"class":84},"'kind'",[71,330,331],{"class":170},", ",[71,333,334],{"class":84},"'v1alpha1'",[71,336,337],{"class":170},")",[71,339,340],{"class":321},"}",[71,342,343],{"class":84},"\u002F",[71,345,322],{"class":321},[71,347,325],{"class":170},[71,349,350],{"class":84},"'name'",[71,352,331],{"class":170},[71,354,355],{"class":84},"''",[71,357,337],{"class":170},[71,359,340],{"class":321},[71,361,362],{"class":84},"\"\n",[71,364,366,369,371],{"class":73,"line":365},33,[71,367,368],{"class":170},"        docs[key] ",[71,370,268],{"class":166},[71,372,373],{"class":170}," doc\n",[71,375,377,380,383,387,389,392,394,397,399,402],{"class":73,"line":376},34,[71,378,379],{"class":166},"    return",[71,381,382],{"class":170}," yaml.safe_dump(docs, ",[71,384,386],{"class":385},"s9osk","sort_keys",[71,388,268],{"class":166},[71,390,391],{"class":321},"True",[71,393,331],{"class":170},[71,395,396],{"class":385},"default_flow_style",[71,398,268],{"class":166},[71,400,401],{"class":321},"False",[71,403,404],{"class":170},").splitlines()\n",[71,406,408],{"class":73,"line":407},35,[71,409,98],{"emptyLinePlaceholder":97},[71,411,413],{"class":73,"line":412},36,[71,414,98],{"emptyLinePlaceholder":97},[71,416,418,420,423],{"class":73,"line":417},37,[71,419,246],{"class":166},[71,421,422],{"class":249}," declared_config",[71,424,425],{"class":170},"(args, node_name):\n",[71,427,429],{"class":73,"line":428},38,[71,430,431],{"class":84},"    \"\"\"Regenerate the node's controlplane config the same way it was created.\"\"\"\n",[71,433,435,438,441,444],{"class":73,"line":434},39,[71,436,437],{"class":166},"    with",[71,439,440],{"class":170}," tempfile.TemporaryDirectory() ",[71,442,443],{"class":166},"as",[71,445,446],{"class":170}," tmp:\n",[71,448,450,453,455,458,460,462,464,466,469,471],{"class":73,"line":449},40,[71,451,452],{"class":170},"        out ",[71,454,268],{"class":166},[71,456,457],{"class":170}," Path(tmp) ",[71,459,343],{"class":166},[71,461,315],{"class":166},[71,463,318],{"class":84},[71,465,322],{"class":321},[71,467,468],{"class":170},"node_name",[71,470,340],{"class":321},[71,472,473],{"class":84},".yaml\"\n",[71,475,477,480,482],{"class":73,"line":476},41,[71,478,479],{"class":170},"        cmd ",[71,481,268],{"class":166},[71,483,484],{"class":170}," [\n",[71,486,488,491,493,496,498,501],{"class":73,"line":487},42,[71,489,490],{"class":84},"            \"talosctl\"",[71,492,331],{"class":170},[71,494,495],{"class":84},"\"gen\"",[71,497,331],{"class":170},[71,499,500],{"class":84},"\"config\"",[71,502,503],{"class":170},", args.cluster_name, args.endpoint,\n",[71,505,507,510],{"class":73,"line":506},43,[71,508,509],{"class":84},"            \"--with-secrets\"",[71,511,512],{"class":170},", args.secrets,\n",[71,514,516,519],{"class":73,"line":515},44,[71,517,518],{"class":84},"            \"--kubernetes-version\"",[71,520,521],{"class":170},", args.kubernetes_version,\n",[71,523,525,528],{"class":73,"line":524},45,[71,526,527],{"class":84},"            \"--talos-version\"",[71,529,530],{"class":170},", args.talos_contract,\n",[71,532,534,537,539,542],{"class":73,"line":533},46,[71,535,536],{"class":84},"            \"--with-docs=false\"",[71,538,331],{"class":170},[71,540,541],{"class":84},"\"--with-examples=false\"",[71,543,544],{"class":170},",\n",[71,546,548,551,553,556],{"class":73,"line":547},47,[71,549,550],{"class":84},"            \"--output-types\"",[71,552,331],{"class":170},[71,554,555],{"class":84},"\"controlplane\"",[71,557,544],{"class":170},[71,559,561,564,566,569],{"class":73,"line":560},48,[71,562,563],{"class":84},"            \"--output\"",[71,565,331],{"class":170},[71,567,568],{"class":321},"str",[71,570,571],{"class":170},"(out),\n",[71,573,575],{"class":73,"line":574},49,[71,576,577],{"class":170},"        ]\n",[71,579,581,584,587,589,592,595,597,600,602,605,608,610,612,614,617],{"class":73,"line":580},50,[71,582,583],{"class":166},"        for",[71,585,586],{"class":170}," patch ",[71,588,283],{"class":166},[71,590,591],{"class":170}," (",[71,593,594],{"class":84},"\"common.yaml\"",[71,596,331],{"class":170},[71,598,599],{"class":84},"\"controlplane.yaml\"",[71,601,331],{"class":170},[71,603,604],{"class":166},"f",[71,606,607],{"class":84},"\"nodes\u002F",[71,609,322],{"class":321},[71,611,468],{"class":170},[71,613,340],{"class":321},[71,615,616],{"class":84},".yaml\"",[71,618,619],{"class":170},"):\n",[71,621,623,626,629,632,635,637,639,642,644,647,649,652,654,656],{"class":73,"line":622},51,[71,624,625],{"class":170},"            cmd ",[71,627,628],{"class":166},"+=",[71,630,631],{"class":170}," [",[71,633,634],{"class":84},"\"--config-patch\"",[71,636,331],{"class":170},[71,638,604],{"class":166},[71,640,641],{"class":84},"\"@",[71,643,322],{"class":321},[71,645,646],{"class":170},"Path(args.patch_dir) ",[71,648,343],{"class":166},[71,650,651],{"class":170}," patch",[71,653,340],{"class":321},[71,655,318],{"class":84},[71,657,658],{"class":170},"]\n",[71,660,662,665,668,670,672,674,677,679,681,683,686,688,690],{"class":73,"line":661},52,[71,663,664],{"class":170},"        subprocess.run(cmd, ",[71,666,667],{"class":385},"check",[71,669,268],{"class":166},[71,671,391],{"class":321},[71,673,331],{"class":170},[71,675,676],{"class":385},"capture_output",[71,678,268],{"class":166},[71,680,391],{"class":321},[71,682,331],{"class":170},[71,684,685],{"class":385},"text",[71,687,268],{"class":166},[71,689,391],{"class":321},[71,691,692],{"class":170},")\n",[71,694,696,699],{"class":73,"line":695},53,[71,697,698],{"class":166},"        return",[71,700,701],{"class":170}," out.read_text()\n",[71,703,705],{"class":73,"line":704},54,[71,706,98],{"emptyLinePlaceholder":97},[71,708,710],{"class":73,"line":709},55,[71,711,98],{"emptyLinePlaceholder":97},[71,713,715,717,720],{"class":73,"line":714},56,[71,716,246],{"class":166},[71,718,719],{"class":249}," live_config",[71,721,722],{"class":170},"(ip):\n",[71,724,726],{"class":73,"line":725},57,[71,727,728],{"class":84},"    \"\"\"Fetch the running config: the .spec of the MachineConfig resource.\"\"\"\n",[71,730,732,735,737],{"class":73,"line":731},58,[71,733,734],{"class":170},"    result ",[71,736,268],{"class":166},[71,738,739],{"class":170}," subprocess.run(\n",[71,741,743,746,749,751,754,757,760,762,765,767,770,772,775,777,780,783,785],{"class":73,"line":742},59,[71,744,745],{"class":170},"        [",[71,747,748],{"class":84},"\"talosctl\"",[71,750,331],{"class":170},[71,752,753],{"class":84},"\"--nodes\"",[71,755,756],{"class":170},", ip, ",[71,758,759],{"class":84},"\"get\"",[71,761,331],{"class":170},[71,763,764],{"class":84},"\"machineconfig\"",[71,766,331],{"class":170},[71,768,769],{"class":84},"\"v1alpha1\"",[71,771,331],{"class":170},[71,773,774],{"class":84},"\"-o\"",[71,776,331],{"class":170},[71,778,779],{"class":84},"\"jsonpath=",[71,781,782],{"class":321},"{.spec}",[71,784,318],{"class":84},[71,786,787],{"class":170},"],\n",[71,789,791,794,796,798,800,802,804,806,808,810,812,814],{"class":73,"line":790},60,[71,792,793],{"class":385},"        check",[71,795,268],{"class":166},[71,797,391],{"class":321},[71,799,331],{"class":170},[71,801,676],{"class":385},[71,803,268],{"class":166},[71,805,391],{"class":321},[71,807,331],{"class":170},[71,809,685],{"class":385},[71,811,268],{"class":166},[71,813,391],{"class":321},[71,815,544],{"class":170},[71,817,819],{"class":73,"line":818},61,[71,820,821],{"class":170},"    )\n",[71,823,825,827],{"class":73,"line":824},62,[71,826,379],{"class":166},[71,828,829],{"class":170}," result.stdout\n",[71,831,833],{"class":73,"line":832},63,[71,834,98],{"emptyLinePlaceholder":97},[71,836,838],{"class":73,"line":837},64,[71,839,98],{"emptyLinePlaceholder":97},[71,841,843,845,848],{"class":73,"line":842},65,[71,844,246],{"class":166},[71,846,847],{"class":249}," main",[71,849,850],{"class":170},"():\n",[71,852,854,857,859,862,865,867,870],{"class":73,"line":853},66,[71,855,856],{"class":170},"    parser ",[71,858,268],{"class":166},[71,860,861],{"class":170}," argparse.ArgumentParser(",[71,863,864],{"class":385},"description",[71,866,268],{"class":166},[71,868,869],{"class":84},"\"Report drift between declared and running Talos configs.\"",[71,871,692],{"class":170},[71,873,875,878,881,883,886,888,890],{"class":73,"line":874},67,[71,876,877],{"class":170},"    parser.add_argument(",[71,879,880],{"class":84},"\"--cluster-name\"",[71,882,331],{"class":170},[71,884,885],{"class":385},"required",[71,887,268],{"class":166},[71,889,391],{"class":321},[71,891,692],{"class":170},[71,893,895,897,900,902,904,906,908,910,913,915,918],{"class":73,"line":894},68,[71,896,877],{"class":170},[71,898,899],{"class":84},"\"--endpoint\"",[71,901,331],{"class":170},[71,903,885],{"class":385},[71,905,268],{"class":166},[71,907,391],{"class":321},[71,909,331],{"class":170},[71,911,912],{"class":385},"help",[71,914,268],{"class":166},[71,916,917],{"class":84},"\"Kubernetes endpoint, e.g. https:\u002F\u002F10.10.20.10:6443\"",[71,919,692],{"class":170},[71,921,923,925,928,930,933,935,938],{"class":73,"line":922},69,[71,924,877],{"class":170},[71,926,927],{"class":84},"\"--secrets\"",[71,929,331],{"class":170},[71,931,932],{"class":385},"default",[71,934,268],{"class":166},[71,936,937],{"class":84},"\"secrets.yaml\"",[71,939,692],{"class":170},[71,941,943,945,948,950,952,954,957],{"class":73,"line":942},70,[71,944,877],{"class":170},[71,946,947],{"class":84},"\"--patch-dir\"",[71,949,331],{"class":170},[71,951,932],{"class":385},[71,953,268],{"class":166},[71,955,956],{"class":84},"\"patches\"",[71,958,692],{"class":170},[71,960,962,964,967,969,971,973,975,977,979,981,984],{"class":73,"line":961},71,[71,963,877],{"class":170},[71,965,966],{"class":84},"\"--kubernetes-version\"",[71,968,331],{"class":170},[71,970,885],{"class":385},[71,972,268],{"class":166},[71,974,391],{"class":321},[71,976,331],{"class":170},[71,978,912],{"class":385},[71,980,268],{"class":166},[71,982,983],{"class":84},"\"Version the cluster runs now\"",[71,985,692],{"class":170},[71,987,989,991,994,996,998,1000,1002,1004,1006,1008,1011],{"class":73,"line":988},72,[71,990,877],{"class":170},[71,992,993],{"class":84},"\"--talos-contract\"",[71,995,331],{"class":170},[71,997,885],{"class":385},[71,999,268],{"class":166},[71,1001,391],{"class":321},[71,1003,331],{"class":170},[71,1005,912],{"class":385},[71,1007,268],{"class":166},[71,1009,1010],{"class":84},"\"Talos version contract used at creation, e.g. v1.14\"",[71,1012,692],{"class":170},[71,1014,1016,1018,1021,1023,1026,1028,1031,1033,1035,1037,1039,1041,1044,1046,1049],{"class":73,"line":1015},73,[71,1017,877],{"class":170},[71,1019,1020],{"class":84},"\"--node\"",[71,1022,331],{"class":170},[71,1024,1025],{"class":385},"action",[71,1027,268],{"class":166},[71,1029,1030],{"class":84},"\"append\"",[71,1032,331],{"class":170},[71,1034,885],{"class":385},[71,1036,268],{"class":166},[71,1038,391],{"class":321},[71,1040,331],{"class":170},[71,1042,1043],{"class":385},"metavar",[71,1045,268],{"class":166},[71,1047,1048],{"class":84},"\"NAME=IP\"",[71,1050,544],{"class":170},[71,1052,1054,1057,1059,1062],{"class":73,"line":1053},74,[71,1055,1056],{"class":385},"                        help",[71,1058,268],{"class":166},[71,1060,1061],{"class":84},"\"Node name (matching patches\u002Fnodes\u002FNAME.yaml) and IP; repeatable\"",[71,1063,692],{"class":170},[71,1065,1067,1070,1072],{"class":73,"line":1066},75,[71,1068,1069],{"class":170},"    args ",[71,1071,268],{"class":166},[71,1073,1074],{"class":170}," parser.parse_args()\n",[71,1076,1078],{"class":73,"line":1077},76,[71,1079,98],{"emptyLinePlaceholder":97},[71,1081,1083,1086,1088],{"class":73,"line":1082},77,[71,1084,1085],{"class":170},"    drifted ",[71,1087,268],{"class":166},[71,1089,1090],{"class":321}," 0\n",[71,1092,1094,1096,1099,1101],{"class":73,"line":1093},78,[71,1095,277],{"class":166},[71,1097,1098],{"class":170}," entry ",[71,1100,283],{"class":166},[71,1102,1103],{"class":170}," args.node:\n",[71,1105,1107,1110,1112,1115,1118,1120,1123],{"class":73,"line":1106},79,[71,1108,1109],{"class":170},"        name, ip ",[71,1111,268],{"class":166},[71,1113,1114],{"class":170}," entry.split(",[71,1116,1117],{"class":84},"\"=\"",[71,1119,331],{"class":170},[71,1121,1122],{"class":321},"1",[71,1124,692],{"class":170},[71,1126,1128,1131],{"class":73,"line":1127},80,[71,1129,1130],{"class":166},"        try",[71,1132,1133],{"class":170},":\n",[71,1135,1137,1140,1142],{"class":73,"line":1136},81,[71,1138,1139],{"class":170},"            declared ",[71,1141,268],{"class":166},[71,1143,1144],{"class":170}," normalize(declared_config(args, name))\n",[71,1146,1148,1151,1153],{"class":73,"line":1147},82,[71,1149,1150],{"class":170},"            live ",[71,1152,268],{"class":166},[71,1154,1155],{"class":170}," normalize(live_config(ip))\n",[71,1157,1159,1162,1165,1167],{"class":73,"line":1158},83,[71,1160,1161],{"class":166},"        except",[71,1163,1164],{"class":170}," subprocess.CalledProcessError ",[71,1166,443],{"class":166},[71,1168,1169],{"class":170}," exc:\n",[71,1171,1173,1176,1179,1181,1184,1186,1189,1191,1194,1196,1199,1202,1205,1208,1210,1213,1215,1218,1220,1222],{"class":73,"line":1172},84,[71,1174,1175],{"class":321},"            print",[71,1177,1178],{"class":170},"(",[71,1180,604],{"class":166},[71,1182,1183],{"class":84},"\"[",[71,1185,322],{"class":321},[71,1187,1188],{"class":170},"name",[71,1190,340],{"class":321},[71,1192,1193],{"class":84},"] ERROR: ",[71,1195,322],{"class":321},[71,1197,1198],{"class":84},"' '",[71,1200,1201],{"class":170},".join(exc.cmd[:",[71,1203,1204],{"class":321},"3",[71,1206,1207],{"class":170},"])",[71,1209,340],{"class":321},[71,1211,1212],{"class":84}," failed: ",[71,1214,322],{"class":321},[71,1216,1217],{"class":170},"exc.stderr.strip()",[71,1219,340],{"class":321},[71,1221,318],{"class":84},[71,1223,692],{"class":170},[71,1225,1227,1230,1232],{"class":73,"line":1226},85,[71,1228,1229],{"class":170},"            drifted ",[71,1231,628],{"class":166},[71,1233,1234],{"class":321}," 1\n",[71,1236,1238],{"class":73,"line":1237},86,[71,1239,304],{"class":166},[71,1241,1243],{"class":73,"line":1242},87,[71,1244,98],{"emptyLinePlaceholder":97},[71,1246,1248,1251,1253,1256,1259,1262,1264,1266,1268,1270,1272,1274,1277,1279,1282,1284,1286,1288,1290,1292,1294,1297,1299,1302,1304,1307],{"class":73,"line":1247},88,[71,1249,1250],{"class":170},"        diff ",[71,1252,268],{"class":166},[71,1254,1255],{"class":321}," list",[71,1257,1258],{"class":170},"(difflib.unified_diff(declared, live, ",[71,1260,1261],{"class":385},"fromfile",[71,1263,268],{"class":166},[71,1265,604],{"class":166},[71,1267,318],{"class":84},[71,1269,322],{"class":321},[71,1271,1188],{"class":170},[71,1273,340],{"class":321},[71,1275,1276],{"class":84}," declared\"",[71,1278,331],{"class":170},[71,1280,1281],{"class":385},"tofile",[71,1283,268],{"class":166},[71,1285,604],{"class":166},[71,1287,318],{"class":84},[71,1289,322],{"class":321},[71,1291,1188],{"class":170},[71,1293,340],{"class":321},[71,1295,1296],{"class":84}," running\"",[71,1298,331],{"class":170},[71,1300,1301],{"class":385},"lineterm",[71,1303,268],{"class":166},[71,1305,1306],{"class":84},"\"\"",[71,1308,1309],{"class":170},"))\n",[71,1311,1313,1315],{"class":73,"line":1312},89,[71,1314,292],{"class":166},[71,1316,1317],{"class":170}," diff:\n",[71,1319,1321,1323,1325],{"class":73,"line":1320},90,[71,1322,1229],{"class":170},[71,1324,628],{"class":166},[71,1326,1234],{"class":321},[71,1328,1330,1332,1334,1336,1338,1340,1342,1344,1347,1349,1352,1354,1357],{"class":73,"line":1329},91,[71,1331,1175],{"class":321},[71,1333,1178],{"class":170},[71,1335,604],{"class":166},[71,1337,1183],{"class":84},[71,1339,322],{"class":321},[71,1341,1188],{"class":170},[71,1343,340],{"class":321},[71,1345,1346],{"class":84},"] DRIFT (",[71,1348,322],{"class":321},[71,1350,1351],{"class":170},"ip",[71,1353,340],{"class":321},[71,1355,1356],{"class":84},"):\"",[71,1358,692],{"class":170},[71,1360,1362,1364,1366,1368,1371,1373],{"class":73,"line":1361},92,[71,1363,1175],{"class":321},[71,1365,1178],{"class":170},[71,1367,318],{"class":84},[71,1369,1370],{"class":321},"\\n",[71,1372,318],{"class":84},[71,1374,1375],{"class":170},".join(diff))\n",[71,1377,1379,1382],{"class":73,"line":1378},93,[71,1380,1381],{"class":166},"        else",[71,1383,1133],{"class":170},[71,1385,1387,1389,1391,1393,1395,1397,1399,1401,1404,1406,1408,1410,1413],{"class":73,"line":1386},94,[71,1388,1175],{"class":321},[71,1390,1178],{"class":170},[71,1392,604],{"class":166},[71,1394,1183],{"class":84},[71,1396,322],{"class":321},[71,1398,1188],{"class":170},[71,1400,340],{"class":321},[71,1402,1403],{"class":84},"] in sync (",[71,1405,322],{"class":321},[71,1407,1351],{"class":170},[71,1409,340],{"class":321},[71,1411,1412],{"class":84},")\"",[71,1414,692],{"class":170},[71,1416,1418],{"class":73,"line":1417},95,[71,1419,98],{"emptyLinePlaceholder":97},[71,1421,1423,1426,1428,1431,1434,1437,1440],{"class":73,"line":1422},96,[71,1424,1425],{"class":170},"    sys.exit(",[71,1427,1122],{"class":321},[71,1429,1430],{"class":166}," if",[71,1432,1433],{"class":170}," drifted ",[71,1435,1436],{"class":166},"else",[71,1438,1439],{"class":321}," 0",[71,1441,692],{"class":170},[71,1443,1445],{"class":73,"line":1444},97,[71,1446,98],{"emptyLinePlaceholder":97},[71,1448,1450],{"class":73,"line":1449},98,[71,1451,98],{"emptyLinePlaceholder":97},[71,1453,1455,1458,1461,1464,1467],{"class":73,"line":1454},99,[71,1456,1457],{"class":166},"if",[71,1459,1460],{"class":321}," __name__",[71,1462,1463],{"class":166}," ==",[71,1465,1466],{"class":84}," \"__main__\"",[71,1468,1133],{"class":170},[71,1470,1472],{"class":73,"line":1471},100,[71,1473,1474],{"class":170},"    main()\n",[11,1476,1477],{},"Run it with the same inputs you used to build the cluster:",[62,1479,1483],{"className":1480,"code":1481,"language":1482,"meta":67,"style":67},"language-bash shiki shiki-themes github-dark","python3 talos_drift_report.py \\\n    --cluster-name lab \\\n    --endpoint https:\u002F\u002F10.10.20.10:6443 \\\n    --kubernetes-version 1.37.0 \\\n    --talos-contract v1.14 \\\n    --node talos-cp1=10.10.20.11 \\\n    --node talos-cp2=10.10.20.12 \\\n    --node talos-cp3=10.10.20.13\n","bash",[26,1484,1485,1496,1506,1516,1526,1536,1549,1561],{"__ignoreMap":67},[71,1486,1487,1490,1493],{"class":73,"line":74},[71,1488,1489],{"class":249},"python3",[71,1491,1492],{"class":84}," talos_drift_report.py",[71,1494,1495],{"class":321}," \\\n",[71,1497,1498,1501,1504],{"class":73,"line":81},[71,1499,1500],{"class":321},"    --cluster-name",[71,1502,1503],{"class":84}," lab",[71,1505,1495],{"class":321},[71,1507,1508,1511,1514],{"class":73,"line":88},[71,1509,1510],{"class":321},"    --endpoint",[71,1512,1513],{"class":84}," https:\u002F\u002F10.10.20.10:6443",[71,1515,1495],{"class":321},[71,1517,1518,1521,1524],{"class":73,"line":94},[71,1519,1520],{"class":321},"    --kubernetes-version",[71,1522,1523],{"class":321}," 1.37.0",[71,1525,1495],{"class":321},[71,1527,1528,1531,1534],{"class":73,"line":101},[71,1529,1530],{"class":321},"    --talos-contract",[71,1532,1533],{"class":84}," v1.14",[71,1535,1495],{"class":321},[71,1537,1538,1541,1544,1547],{"class":73,"line":107},[71,1539,1540],{"class":321},"    --node",[71,1542,1543],{"class":84}," talos-cp1=",[71,1545,1546],{"class":321},"10.10.20.11",[71,1548,1495],{"class":321},[71,1550,1551,1553,1556,1559],{"class":73,"line":113},[71,1552,1540],{"class":321},[71,1554,1555],{"class":84}," talos-cp2=",[71,1557,1558],{"class":321},"10.10.20.12",[71,1560,1495],{"class":321},[71,1562,1563,1565,1568],{"class":73,"line":119},[71,1564,1540],{"class":321},[71,1566,1567],{"class":84}," talos-cp3=",[71,1569,1570],{"class":321},"10.10.20.13\n",[62,1572,1576],{"className":1573,"code":1575,"language":685,"meta":67},[1574],"language-text","[talos-cp1] in sync (10.10.20.11)\n[talos-cp2] DRIFT (10.10.20.12):\n--- talos-cp2 declared\n+++ talos-cp2 running\n@@ -41,6 +41,8 @@\n   kind: KubeNodeConfig\n   labels:\n     node-role.kubernetes.io\u002Fcontrol-plane: ''\n+    topology.kubernetes.io\u002Fzone: rack-b\n[talos-cp3] in sync (10.10.20.13)\n",[26,1577,1575],{"__ignoreMap":67},[11,1579,1580,1581,1584,1585,1588,1589,1592],{},"In that example someone ran ",[26,1582,1583],{},"talosctl patch machineconfig"," to add a label and never wrote a patch file for it. The fix is a one-line patch in Git, not an archaeology session. Pass the Kubernetes version the cluster runs ",[45,1586,1587],{},"now",": ",[26,1590,1591],{},"talosctl upgrade-k8s"," updates component images in the live configs, which is exactly the kind of drift the reproducible-configuration guide warns about if you regenerate with an older version.",[11,1594,1595,1596,1601],{},"The layout and bootstrap flow this assumes are in ",[1597,1598,1600],"a",{"href":1599},"\u002F2026\u002F03\u002F18\u002Ftalos-linux-bootstrapping-a-three-node-kubernetes-cluster-from-bare-metal\u002F","Talos Linux: Bootstrapping a Three-Node Cluster on Bare Metal",".",[18,1603,1605],{"id":1604},"where-it-actually-saves-operational-time","Where it actually saves operational time",[11,1607,1608,1609,1612,1613,1616],{},"Patching is the concrete thing people ask about. Upgrading Talos is an API call that hands the node an installer image. The node cordons and drains itself, stops its services, unmounts its filesystems, writes the new image, and sets the bootloader to boot the new version ",[45,1610,1611],{},"once",". Only after it comes back and verifies itself does it make that choice permanent, rejoin, and uncordon. Upgrades use an A-B scheme that keeps the previous kernel and OS image, so a node that fails to boot the new version falls back to the old one on the next reboot. If the node boots fine but your workloads don't like it, ",[26,1614,1615],{},"talosctl rollback"," switches back to the previous image. There's no apt or yum transaction that can half-complete, and no reboot that comes back into a broken initramfs because a kernel module got orphaned. Talos also refuses to upgrade a control plane node when doing so would cost etcd quorum. I haven't had a single \"the patch broke the box and now I'm on the iDRAC console at midnight\" incident on Talos nodes. On traditional distros at similar fleet sizes that was a recurring event, and it's a category of pager alert that simply stopped.",[11,1618,1619,1620,1623],{},"Configuration changes got safer in a way I didn't expect. In Talos 1.14, config changes are applied to the running node without a reboot. ",[26,1621,1622],{},"--mode=try"," applies a change and reverts it automatically after a timeout (one minute by default) if no further configuration update is applied. That's the right tool for anything that could cut off your own access, like a network change. I use it as a rehearsal: try the change, check from a second session that the node is still reachable and healthy, let it revert, then apply it for real:",[62,1625,1627],{"className":1480,"code":1626,"language":1482,"meta":67,"style":67},"# Rehearse an MTU change; Talos reverts it after 2 minutes\ntalosctl -n 10.10.20.12 patch machineconfig --mode=try --timeout 2m -p @patches\u002Fmtu-9000.yaml\n\n# Preview what the real change will do, then apply it in the default mode\ntalosctl -n 10.10.20.12 patch machineconfig --dry-run -p @patches\u002Fmtu-9000.yaml\ntalosctl -n 10.10.20.12 patch machineconfig -p @patches\u002Fmtu-9000.yaml\n",[26,1628,1629,1634,1665,1669,1674,1693],{"__ignoreMap":67},[71,1630,1631],{"class":73,"line":74},[71,1632,1633],{"class":77},"# Rehearse an MTU change; Talos reverts it after 2 minutes\n",[71,1635,1636,1639,1642,1645,1647,1650,1653,1656,1659,1662],{"class":73,"line":81},[71,1637,1638],{"class":249},"talosctl",[71,1640,1641],{"class":321}," -n",[71,1643,1644],{"class":321}," 10.10.20.12",[71,1646,651],{"class":84},[71,1648,1649],{"class":84}," machineconfig",[71,1651,1652],{"class":321}," --mode=try",[71,1654,1655],{"class":321}," --timeout",[71,1657,1658],{"class":84}," 2m",[71,1660,1661],{"class":321}," -p",[71,1663,1664],{"class":84}," @patches\u002Fmtu-9000.yaml\n",[71,1666,1667],{"class":73,"line":88},[71,1668,98],{"emptyLinePlaceholder":97},[71,1670,1671],{"class":73,"line":94},[71,1672,1673],{"class":77},"# Preview what the real change will do, then apply it in the default mode\n",[71,1675,1676,1678,1680,1682,1684,1686,1689,1691],{"class":73,"line":101},[71,1677,1638],{"class":249},[71,1679,1641],{"class":321},[71,1681,1644],{"class":321},[71,1683,651],{"class":84},[71,1685,1649],{"class":84},[71,1687,1688],{"class":321}," --dry-run",[71,1690,1661],{"class":321},[71,1692,1664],{"class":84},[71,1694,1695,1697,1699,1701,1703,1705,1707],{"class":73,"line":107},[71,1696,1638],{"class":249},[71,1698,1641],{"class":321},[71,1700,1644],{"class":321},[71,1702,651],{"class":84},[71,1704,1649],{"class":84},[71,1706,1661],{"class":321},[71,1708,1664],{"class":84},[11,1710,1711],{},"The patch itself is an ordinary document:",[62,1713,1717],{"className":1714,"code":1715,"language":1716,"meta":67,"style":67},"language-yaml shiki shiki-themes github-dark","apiVersion: v1alpha1\nkind: LinkConfig\nname: net0\nmtu: 9000\n","yaml",[26,1718,1719,1730,1740,1749],{"__ignoreMap":67},[71,1720,1721,1725,1727],{"class":73,"line":74},[71,1722,1724],{"class":1723},"s4JwU","apiVersion",[71,1726,1588],{"class":170},[71,1728,1729],{"class":84},"v1alpha1\n",[71,1731,1732,1735,1737],{"class":73,"line":81},[71,1733,1734],{"class":1723},"kind",[71,1736,1588],{"class":170},[71,1738,1739],{"class":84},"LinkConfig\n",[71,1741,1742,1744,1746],{"class":73,"line":88},[71,1743,1188],{"class":1723},[71,1745,1588],{"class":170},[71,1747,1748],{"class":84},"net0\n",[71,1750,1751,1754,1756],{"class":73,"line":94},[71,1752,1753],{"class":1723},"mtu",[71,1755,1588],{"class":170},[71,1757,1758],{"class":321},"9000\n",[11,1760,1761,1762,1768,1769,1772],{},"Talos' ",[1597,1763,1767],{"href":1764,"rel":1765},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fconfigure-your-talos-cluster\u002Fsystem-configuration\u002Fpatching",[1766],"nofollow","configuration patching docs"," are explicit that \"Talos supports patching multi-document machine configuration,\" and list ",[26,1770,1771],{},"talosctl patch"," as the way to patch a running node's configuration, so a per-topic document like this one is a complete patch on its own.",[11,1774,1775,1776,1779],{},"One caveat from the same guide: a few settings are only read when a long-running service starts. etcd settings are the notable one, and Talos deliberately doesn't restart etcd on a config change, so those still need an explicit ",[26,1777,1778],{},"talosctl reboot",", sequenced like any other control plane maintenance.",[18,1781,1783],{"id":1782},"where-the-trade-off-actually-bites","Where the trade-off actually bites",[11,1785,1786,1787,1790],{},"The cost is that every troubleshooting habit built over a career of ",[26,1788,1789],{},"ssh"," and grep stops working. When a node is behaving strangely you don't tail a log file. You pull what you need through the API, and it pays to have the equivalents memorized before the incident, not during it:",[62,1792,1794],{"className":1480,"code":1793,"language":1482,"meta":67,"style":67},"# Text UI with node overview, logs and real-time metrics\ntalosctl -n 10.10.20.12 dashboard\n\n# Service logs (Talos services), and Kubernetes container logs via the cri namespace\ntalosctl -n 10.10.20.12 logs kubelet --tail 200\ntalosctl -n 10.10.20.12 containers --namespace cri\ntalosctl -n 10.10.20.12 logs --namespace cri \"$CONTAINER_ID\"\n\n# Kernel log, streaming\ntalosctl -n 10.10.20.12 dmesg --follow\n\n# Service state, including etcd on control plane nodes\ntalosctl -n 10.10.20.12 service etcd\n\n# Listening sockets with owning processes (note: -n is --nodes in talosctl, not \"numeric\")\ntalosctl -n 10.10.20.12 netstat --tcp --listening --programs\n\n# Packet capture streamed to your workstation's tcpdump (physical link name from talosctl get links)\ntalosctl -n 10.10.20.12 pcap --interface enp1s0 --duration 30s -o - | tcpdump -nn -r -\n\n# Read a file under \u002Fproc or \u002Fsys\ntalosctl -n 10.10.20.12 read \u002Fproc\u002Fcmdline\n\n# Everything above plus COSI resources, bundled for a support case\ntalosctl -n 10.10.20.12 support --output support-talos-cp2.zip\n",[26,1795,1796,1801,1812,1816,1821,1841,1858,1881,1885,1890,1904,1908,1913,1927,1931,1936,1956,1960,1965,2009,2013,2018,2032,2036,2041],{"__ignoreMap":67},[71,1797,1798],{"class":73,"line":74},[71,1799,1800],{"class":77},"# Text UI with node overview, logs and real-time metrics\n",[71,1802,1803,1805,1807,1809],{"class":73,"line":81},[71,1804,1638],{"class":249},[71,1806,1641],{"class":321},[71,1808,1644],{"class":321},[71,1810,1811],{"class":84}," dashboard\n",[71,1813,1814],{"class":73,"line":88},[71,1815,98],{"emptyLinePlaceholder":97},[71,1817,1818],{"class":73,"line":94},[71,1819,1820],{"class":77},"# Service logs (Talos services), and Kubernetes container logs via the cri namespace\n",[71,1822,1823,1825,1827,1829,1832,1835,1838],{"class":73,"line":101},[71,1824,1638],{"class":249},[71,1826,1641],{"class":321},[71,1828,1644],{"class":321},[71,1830,1831],{"class":84}," logs",[71,1833,1834],{"class":84}," kubelet",[71,1836,1837],{"class":321}," --tail",[71,1839,1840],{"class":321}," 200\n",[71,1842,1843,1845,1847,1849,1852,1855],{"class":73,"line":107},[71,1844,1638],{"class":249},[71,1846,1641],{"class":321},[71,1848,1644],{"class":321},[71,1850,1851],{"class":84}," containers",[71,1853,1854],{"class":321}," --namespace",[71,1856,1857],{"class":84}," cri\n",[71,1859,1860,1862,1864,1866,1868,1870,1873,1876,1879],{"class":73,"line":113},[71,1861,1638],{"class":249},[71,1863,1641],{"class":321},[71,1865,1644],{"class":321},[71,1867,1831],{"class":84},[71,1869,1854],{"class":321},[71,1871,1872],{"class":84}," cri",[71,1874,1875],{"class":84}," \"",[71,1877,1878],{"class":170},"$CONTAINER_ID",[71,1880,362],{"class":84},[71,1882,1883],{"class":73,"line":119},[71,1884,98],{"emptyLinePlaceholder":97},[71,1886,1887],{"class":73,"line":125},[71,1888,1889],{"class":77},"# Kernel log, streaming\n",[71,1891,1892,1894,1896,1898,1901],{"class":73,"line":130},[71,1893,1638],{"class":249},[71,1895,1641],{"class":321},[71,1897,1644],{"class":321},[71,1899,1900],{"class":84}," dmesg",[71,1902,1903],{"class":321}," --follow\n",[71,1905,1906],{"class":73,"line":136},[71,1907,98],{"emptyLinePlaceholder":97},[71,1909,1910],{"class":73,"line":141},[71,1911,1912],{"class":77},"# Service state, including etcd on control plane nodes\n",[71,1914,1915,1917,1919,1921,1924],{"class":73,"line":147},[71,1916,1638],{"class":249},[71,1918,1641],{"class":321},[71,1920,1644],{"class":321},[71,1922,1923],{"class":84}," service",[71,1925,1926],{"class":84}," etcd\n",[71,1928,1929],{"class":73,"line":153},[71,1930,98],{"emptyLinePlaceholder":97},[71,1932,1933],{"class":73,"line":158},[71,1934,1935],{"class":77},"# Listening sockets with owning processes (note: -n is --nodes in talosctl, not \"numeric\")\n",[71,1937,1938,1940,1942,1944,1947,1950,1953],{"class":73,"line":163},[71,1939,1638],{"class":249},[71,1941,1641],{"class":321},[71,1943,1644],{"class":321},[71,1945,1946],{"class":84}," netstat",[71,1948,1949],{"class":321}," --tcp",[71,1951,1952],{"class":321}," --listening",[71,1954,1955],{"class":321}," --programs\n",[71,1957,1958],{"class":73,"line":174},[71,1959,98],{"emptyLinePlaceholder":97},[71,1961,1962],{"class":73,"line":182},[71,1963,1964],{"class":77},"# Packet capture streamed to your workstation's tcpdump (physical link name from talosctl get links)\n",[71,1966,1967,1969,1971,1973,1976,1979,1982,1985,1988,1991,1994,1997,2000,2003,2006],{"class":73,"line":190},[71,1968,1638],{"class":249},[71,1970,1641],{"class":321},[71,1972,1644],{"class":321},[71,1974,1975],{"class":84}," pcap",[71,1977,1978],{"class":321}," --interface",[71,1980,1981],{"class":84}," enp1s0",[71,1983,1984],{"class":321}," --duration",[71,1986,1987],{"class":84}," 30s",[71,1989,1990],{"class":321}," -o",[71,1992,1993],{"class":84}," -",[71,1995,1996],{"class":166}," |",[71,1998,1999],{"class":249}," tcpdump",[71,2001,2002],{"class":321}," -nn",[71,2004,2005],{"class":321}," -r",[71,2007,2008],{"class":84}," -\n",[71,2010,2011],{"class":73,"line":198},[71,2012,98],{"emptyLinePlaceholder":97},[71,2014,2015],{"class":73,"line":206},[71,2016,2017],{"class":77},"# Read a file under \u002Fproc or \u002Fsys\n",[71,2019,2020,2022,2024,2026,2029],{"class":73,"line":220},[71,2021,1638],{"class":249},[71,2023,1641],{"class":321},[71,2025,1644],{"class":321},[71,2027,2028],{"class":84}," read",[71,2030,2031],{"class":84}," \u002Fproc\u002Fcmdline\n",[71,2033,2034],{"class":73,"line":225},[71,2035,98],{"emptyLinePlaceholder":97},[71,2037,2038],{"class":73,"line":233},[71,2039,2040],{"class":77},"# Everything above plus COSI resources, bundled for a support case\n",[71,2042,2043,2045,2047,2049,2052,2055],{"class":73,"line":238},[71,2044,1638],{"class":249},[71,2046,1641],{"class":321},[71,2048,1644],{"class":321},[71,2050,2051],{"class":84}," support",[71,2053,2054],{"class":321}," --output",[71,2056,2057],{"class":84}," support-talos-cp2.zip\n",[11,2059,2060,2061,2064,2065,2068,2069,2071,2072,2075,2076,2079],{},"If the thing you need to inspect isn't exposed, you're stuck, or at least slowed down. I hit exactly this with a NIC firmware quirk that would have been a five-minute ",[26,2062,2063],{},"ethtool"," session on a normal box and instead took an afternoon of working around the API's more limited surface. The surface has grown since then. ",[26,2066,2067],{},"talosctl get ethernetstatus \u003Clink> -o yaml"," shows ring sizes and offload features in ",[26,2070,2063],{}," terms, and the ",[26,2073,2074],{},"EthernetConfig"," document can change them declaratively. ",[26,2077,2078],{},"talosctl debug \u003Cimage>"," runs a debug container on the node from an image reference or a local tarball, so you can bring your own tools for the rare case that needs them. Any team adopting Talos still needs to budget real time for this relearning, and needs to accept that a certain class of \"let me just poke at it directly\" debugging is gone for good, not just discouraged.",[18,2081,2083],{"id":2082},"where-id-actually-recommend-it","Where I'd actually recommend it",[11,2085,2086],{},"Talos earns its keep on infrastructure where the node's only job is to run Kubernetes: bare metal or VM pools dedicated to a cluster, no other tenancy, no other reason for a human to touch the box directly. It's a worse fit for mixed-use infrastructure where a node also needs to run something outside Kubernetes, or for a team that isn't already comfortable operating Kubernetes without SSH-shaped crutches. The immutability isn't a security feature bolted onto a Linux distro. It's a constraint the whole operating model is built around, and it only pays off when the rest of your operations already fit that constraint.",[18,2088,2090],{"id":2089},"the-honest-tradeoff","The honest tradeoff",[11,2092,2093,2094,2096],{},"Immutable infrastructure doesn't remove operational risk, it relocates it: from \"did the sysadmin's manual fix get documented\" to \"do our secrets bundle and patches in version control actually represent what we want running.\" That's a better place for the risk to live, and the drift report above makes it measurable, but it still needs the same discipline: patch review, staged rollouts, a backed-up ",[26,2095,55],{},", and someone who owns the machine config the way they used to own the runbook. Talos didn't make operations easier so much as it made the easy parts easier and the hard parts more honest about being hard.",[18,2098,2100],{"id":2099},"references","References",[2102,2103,2104,2113,2121,2132,2139,2145,2152],"ul",{},[2105,2106,2107,2112],"li",{},[1597,2108,2111],{"href":2109,"rel":2110},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Flearn-more\u002Fphilosophy",[1766],"Philosophy"," (SquashFS root, no shell or SSH, ephemeral partition)",[2105,2114,2115,2120],{},[1597,2116,2119],{"href":2117,"rel":2118},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fconfigure-your-talos-cluster\u002Flifecycle-management\u002Fupgrading-talos",[1766],"Upgrading Talos Linux"," (A-B images, upgrade sequence, rollback, quorum protection)",[2105,2122,2123,2128,2129,2131],{},[1597,2124,2127],{"href":2125,"rel":2126},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fconfigure-your-talos-cluster\u002Fsystem-configuration\u002Fediting-machine-configuration",[1766],"Edit Machine Configuration"," (apply modes, ",[26,2130,1622],{},", changes that still need a reboot)",[2105,2133,2134],{},[1597,2135,2138],{"href":2136,"rel":2137},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fconfigure-your-talos-cluster\u002Fsystem-configuration\u002Freproducible-machine-configuration",[1766],"Reproducible Machine Configuration",[2105,2140,2141],{},[1597,2142,2144],{"href":1764,"rel":2143},[1766],"Configuration Patches",[2105,2146,2147],{},[1597,2148,2151],{"href":2149,"rel":2150},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Fnetworking\u002Fadvanced\u002Fethernet-config",[1766],"Ethernet Configuration",[2105,2153,2154,591,2159,331,2162,331,2165,331,2168,331,2171,331,2174,331,2177,331,2180,337],{},[1597,2155,2158],{"href":2156,"rel":2157},"https:\u002F\u002Fdocs.siderolabs.com\u002Ftalos\u002Fv1.14\u002Freference\u002Fcli",[1766],"talosctl CLI reference",[26,2160,2161],{},"dashboard",[26,2163,2164],{},"logs",[26,2166,2167],{},"dmesg",[26,2169,2170],{},"netstat",[26,2172,2173],{},"pcap",[26,2175,2176],{},"read",[26,2178,2179],{},"debug",[26,2181,2182],{},"support",[2184,2185,2186],"style",{},"html pre.shiki code .sAwPA, html code.shiki .sAwPA{--shiki-default:#6A737D}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .svObZ, html code.shiki .svObZ{--shiki-default:#B392F0}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .s9osk, html code.shiki .s9osk{--shiki-default:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .s4JwU, html code.shiki .s4JwU{--shiki-default:#85E89D}",{"title":67,"searchDepth":81,"depth":81,"links":2188},[2189,2190,2191,2192,2193,2194],{"id":20,"depth":81,"text":21},{"id":1604,"depth":81,"text":1605},{"id":1782,"depth":81,"text":1783},{"id":2082,"depth":81,"text":2083},{"id":2089,"depth":81,"text":2090},{"id":2099,"depth":81,"text":2100},"techcolumnist",[2197,2198],"engineering","strategy","2026-03-25T14:00:00Z","Where an immutable, API-managed OS pays off under Kubernetes and where it bites: drift, upgrades, try-mode config changes, and debugging without SSH.","md",false,null,{},"\u002Fblog\u002F2026\u002F03\u002F25\u002Ftalos-linux-what-immutable-infrastructure-actually-buys-you-in-production",{"title":6,"description":2200},[2195],"blog\u002F2026\u002F03\u002F25\u002Ftalos-linux-what-immutable-infrastructure-actually-buys-you-in-production",[2210,2211],"talos","kubernetes","2026-09-16T07:22:17Z","\u002F2026\u002F03\u002F25\u002Ftalos-linux-what-immutable-infrastructure-actually-buys-you-in-production\u002F","e2JnJakv9dtGrZYJDr8bfIoq0t-gd_8eYMZHUoKtW5Y",{"title":2216,"description":2217,"date":2218,"url":2219,"categories":2220,"tags":2221,"image":2203,"readingTime":113,"canonical":2195,"sites":2226,"series":2203,"seriesOrder":2203},"Intune: Migrating From Group Policy to Cloud-Native Management","Moving a Windows estate from Group Policy to Intune: GPO inventory script, Group Policy analytics, Settings catalog migration, MDMWinsOverGP and conflict checks.","2026-04-01T14:00:00Z","\u002F2026\u002F04\u002F01\u002Fintune-migrating-from-group-policy-to-cloud-native-endpoint-management\u002F",[2197,2198],[2222,2223,2224,2225],"intune","gpo","windows","active-directory",[2195],{"title":1600,"description":2228,"date":2229,"url":1599,"categories":2230,"tags":2231,"image":2203,"readingTime":113,"canonical":2195,"sites":2233,"series":2203,"seriesOrder":2203},"Bringing up a three-node Talos Kubernetes cluster on bare metal: secrets, per-node patches, a Layer 2 VIP, apply-config, a single bootstrap, and a full script.","2026-03-18T14:00:00Z",[2197],[2210,2211,2232],"cluster",[2195],[2235,2244,2252],{"title":2236,"description":2237,"date":2238,"url":2239,"categories":2240,"tags":2241,"image":2203,"readingTime":101,"canonical":2195,"sites":2243,"series":2203,"seriesOrder":2203},"Hyper-V to Talos: Rethinking the Virtualization Stack","Why moving workloads off Hyper-V VMs onto a Talos Kubernetes cluster changed the whole stack, with an inventory script and a VM-to-manifest example.","2026-09-16T14:00:00Z","\u002F2026\u002F09\u002F16\u002Fhyper-v-to-talos-rethinking-the-virtualization-stack\u002F",[2197,2198],[2242,2210,2211],"hyper-v",[2195],{"title":2245,"description":2246,"date":2247,"url":2248,"categories":2249,"tags":2250,"image":2203,"readingTime":113,"canonical":2195,"sites":2251,"series":2203,"seriesOrder":2203},"Talos Linux: Upgrading a Cluster Without a Maintenance Window","Rolling Talos and Kubernetes upgrades through a cluster one node at a time: preflight checks, etcd snapshots, Image Factory installers, and scripts.","2026-08-26T14:00:00Z","\u002F2026\u002F08\u002F26\u002Ftalos-linux-upgrading-a-cluster-without-a-maintenance-window\u002F",[2197],[2210,2211],[2195],{"title":2253,"description":2254,"date":2255,"url":2256,"categories":2257,"tags":2258,"image":2203,"readingTime":113,"canonical":2195,"sites":2259,"series":2203,"seriesOrder":2203},"Kubernetes: Running Talos Alongside a Legacy Hyper-V Estate","Introducing a Talos Kubernetes cluster into an existing Hyper-V estate: VM build script, Image Factory schematic, VLAN, VIP, firewall and storage boundaries.","2026-07-08T14:00:00Z","\u002F2026\u002F07\u002F08\u002Fkubernetes-running-talos-alongside-a-legacy-hyper-v-estate\u002F",[2197],[2210,2211,2242],[2195],{"doc":2203,"posts":2261},[],1790052513818]