tom@techcolumnist:~$

~/2026/07/01/windows-server-tcp-stack-tuning-for-a-10gbe-storage-network.md

Windows Server: TCP Stack Tuning for a 10GbE Storage Network

---
author: 
date: 
read: 7 min
in:   [engineering]
tags: [windows, tcp, network]
---

$ grep -n '^#' post.md

A 10GbE link between a Windows Server host and an iSCSI or SMB storage target should, in theory, move a little over a gigabyte a second. In practice, the box you just cabled up will often plateau well below that, and the reflex is to blame the switch, the cabling, or the storage array before anyone looks at the host's own network stack. I've chased that plateau enough times now to check the host first, because more often than not that's where the ceiling is. Most of the time it isn't a missing registry tweak, either. It's one of a handful of settings that Microsoft documents clearly and that have drifted from where they should be.

Start with a baseline, not a setting

The tuning below only matters if you know what "good" looks like on that specific link. I always run a raw throughput test between the two endpoints with no storage protocol in the way before touching anything, so a marginal cable or a port negotiated at the wrong speed doesn't get misdiagnosed as a stack problem three settings into a change window. Microsoft's NTTTCP tool is what I use between Windows hosts. The receiver and sender commands differ only in -r and -s. -m takes threads, CPU (* for any) and the receiver's IP; Microsoft's own guidance uses twice the receiver's core count for threads, so the 16 below is for an eight-core receiver. -t is the duration in seconds:

powershell
# On the receiver (the storage-side host).
ntttcp.exe -r -m 16,*,10.20.30.11 -t 60

# On the sender.
ntttcp.exe -s -m 16,*,10.20.30.11 -t 60

Run it once with -m 1,*,<ip> too. The gap between one thread and many is the whole story of the next section.

One connection lands on one core

Receive Side Scaling exists because a single CPU core can't keep up with a fast NIC. RSS hashes each incoming packet's headers and spreads different connections across cores. The key word is connections. Microsoft's SMB Multichannel documentation spells out the consequence for storage: with a single 10GbE adapter and one SMB session without Multichannel, "SMB creates a single TCP/IP connection. With only a single CPU core, this configuration inherently leads to congestion, especially when many small I/O operations are performed," and "if you use a single connection, RSS cannot help."

So there are two things to check, in order:

  1. RSS is on and has cores to use. Get-NetAdapterRss shows Enabled, the Profile (the default is NUMAStatic), BaseProcessorNumber and MaxProcessors. I've found RSS enabled at the driver level while every one of its queues was pinned to core 0, and I've found it switched off globally (Get-NetOffloadGlobalSetting); "enabled" in the adapter properties proves neither. The full object (Get-NetAdapterRss -Name <adapter> | Format-List -Property *) shows which processors are actually in the RSS array. Microsoft's own advice on the knobs is restrained: "Selecting the correct profile should be sufficient in most scenarios." Set BaseProcessorNumber and MaxProcessors only when you're deliberately dividing cores between several adapters. If Task Manager shows logical processors underused for receive traffic, the tuning guide suggests raising the number of RSS queues toward what the adapter supports.
  2. The storage protocol actually uses more than one connection. For SMB 3.x that's SMB Multichannel, which is on by default and, with an RSS-capable NIC, "creates multiple TCP/IP connections for that particular session." Two traps: Multichannel ignores slower adapters when a faster one exists ("network adapters that are the same type... and have the same speed are simultaneously used"), and disabling it on either the client or the server disables it for both. For iSCSI, the equivalent is multiple sessions or connections through MPIO, which is a separate configuration on the initiator.

Verify Multichannel during a long copy, not at idle:

powershell
Get-SmbClientConfiguration | Select-Object -Property EnableMultiChannel
Get-SmbClientNetworkInterface
Get-SmbMultichannelConnection

If Multichannel isn't selecting an interface, Microsoft's troubleshooting guide checks the obvious first: ms_server and ms_msclient bound on the adapter (Get-NetAdapterBinding -ComponentID ms_server,ms_msclient), the interface listed by Get-SmbServerNetworkInterface, the network profile not being Public, and nothing in the path blocking the SMB3 NETWORK_INTERFACE_INFO exchange. SMB2 and earlier don't do Multichannel at all.

The TCP window is autotuned; leave it that way

It's tempting to decide that small, low-latency storage I/O never gives autotuning a fair chance and to pin the window instead. I've been down that road, and it isn't what Microsoft documents. Receive window autotuning has five levels (Normal, Disabled, Restricted, HighlyRestricted, Experimental) and Normal is the default, described as growing the window "to accommodate almost all scenarios." Disabled is the one that hurts: the window stays at its default size, which depends on link speed (64 KB from 100 Mbps up to 10 Gbps, 128 KB at 10 Gbps and above). Microsoft's formula is throughput = window / latency, so a fixed window caps a single connection no matter how fast the wire is.

The other half of the story is that the old registry method is gone. Microsoft lists TcpWindowSize, NumTcbTablePartitions and MaxHashTableSize under HKLM\System\CurrentControlSet\Services\Tcpip\Parameters as "no longer supported, and are ignored," and "starting with Windows Server 2019, you can no longer use the registry to configure the TCP receive window size." If a tuning guide tells you to create those values, it was written for Windows Server 2003. Check and set autotuning with the supported tools:

powershell
Get-NetTCPSetting | Select-Object -Property SettingName, AutoTuningLevelLocal, CongestionProvider
Set-NetTCPSetting -AutoTuningLevelLocal Normal

Same for congestion control: since Windows Server 2019, the Internet, Datacenter and Custom templates use CUBIC (Compat stays on NewReno). DCTCP is only worth considering when the switches mark ECN end to end, which is a fabric project, not a host setting.

Offloads help until one of them doesn't

Checksum offload, Large Send Offload and Receive Segment Coalescing reduce CPU cost per byte by pushing work onto the NIC, which matters at 10GbE rates. Microsoft's guidance: "Enabling network adapter offload features is usually beneficial," with the caveat that a NIC without enough hardware resources can sustain less throughput with segmentation offload on. The two offloads Microsoft says not to use at all are TCP Chimney Offload and IPsec Task Offload: "deprecated in Windows Server 2016, and might adversely affect server and networking performance." If you see Chimney in an old tuning checklist, that checklist predates Server 2016.

RSC deserves a specific check because it can be enabled and still not be working. Get-NetAdapterRsc reports IPv4OperationalState and, if it's off, IPv4FailureReason: documented reasons include a Windows Filtering Platform filter in the path (WFPCompatibility), an old driver (NDISCompatibility), IP forwarding being on (ForwardingEnabled), or RSC disabled globally (NetOffloadGlobalDisabled). WFP is the one that surprises people: the tuning guide warns that "a poorly written WFP filter can significantly decrease a server's networking performance," and firewall and antivirus products are the usual source.

The catch with offloads is the NIC doing the work. Microsoft's network adapter tuning guide only goes as far as the resource caveat above ("the network adapter might not be powerful enough to handle the offload capabilities with high throughput"); it says nothing about firmware bugs. In my experience, though, offload bugs in NIC firmware and drivers turn up often enough that "disable offloads and see if it gets faster" is a legitimate diagnostic step, not folklore. I've hit at least one NIC generation where LSO corrupted iSCSI frames under sustained load in a way that only showed up as intermittent retransmits, not outright failures, the kind of problem that survives a quick smoke test. That's why the baseline script below records the retransmit rate.

Jumbo frames only pay off end to end

Enabling 9000-byte jumbo frames on the server NIC without confirming every switch port and the storage target's NIC match doesn't just fail to help. The oversized frames get dropped or fragmented, which looks like intermittent packet loss under load rather than a configuration mismatch. On Windows, set it with the standardized driver keyword rather than a vendor display name. *JumboPacket is defined as the largest frame the hardware supports, counted including the Ethernet header, with a default of 1514 (a 1500-byte MTU), so a 9000-byte MTU is 9014. The valid range is vendor-defined, so check it first:

powershell
Get-NetAdapterAdvancedProperty -Name "SLOT 2 Port 1" -RegistryKeyword "*JumboPacket" | Format-List -Property DisplayValue, RegistryValue, ValidRegistryValues
Set-NetAdapterAdvancedProperty -Name "SLOT 2 Port 1" -RegistryKeyword "*JumboPacket" -RegistryValue 9014

Then test the path, rather than trusting that three management consoles agree. ping -f sets Don't Fragment, and -l is the payload: 9000 minus 28 bytes of IPv4 and ICMP headers is 8972.

powershell
ping.exe -f -l 8972 10.20.30.11

A baseline script for the host

Before and after any change I capture the same snapshot on both ends, so "it's faster" is a comparison and not a feeling. This script is read-only. It records RSS, RSC and jumbo state per adapter and the host-wide offload state, flags ignored legacy registry values and the Chimney state, lists SMB Multichannel connections, and samples the TCP retransmit rate while you run a copy or NTTTCP. It needs Windows Server 2016 or later and an elevated session.

powershell
<#
.SYNOPSIS
    Captures a read-only network baseline for storage-facing adapters on Windows Server.
.DESCRIPTION
    Reports RSS, RSC, *JumboPacket and global offload state for the named adapters,
    TCP autotuning and congestion provider per template, legacy Tcpip registry
    values that Windows ignores, current SMB Multichannel connections, and the
    TCPv4 retransmit rate sampled over a short window. Changes nothing.
.PARAMETER AdapterName
    Storage-facing adapter names, as shown by Get-NetAdapter.
.PARAMETER SampleSeconds
    How long to sample TCP counters. Default 30. Run a copy or NTTTCP meanwhile.
.EXAMPLE
    .\Get-StorageNicBaseline.ps1 -AdapterName "SLOT 2 Port 1", "SLOT 2 Port 2" -SampleSeconds 60
.NOTES
    Author  : Thomas Lasswell (https://www.techcolumnist.com)
    Version : 1.0 (2026-07-01)
    Requires: Windows Server 2016 or later, NetAdapter, NetTCPIP and SmbShare modules
#>
[CmdletBinding()]
param (
    [Parameter(Mandatory = $true)]
    [string[]]$AdapterName,

    [ValidateRange(5, 600)]
    [int]$SampleSeconds = 30
)

Write-Host "== Adapters"
Get-NetAdapter -Name $AdapterName |
    Format-Table -Property Name, InterfaceDescription, Status, LinkSpeed, DriverVersion -AutoSize

Write-Host "== RSS"
Get-NetAdapterRss -Name $AdapterName |
    Format-Table -Property Name, Enabled, Profile, BaseProcessorNumber, MaxProcessors, NumberOfReceiveQueues -AutoSize

Write-Host "== RSC (enabled vs actually operating)"
Get-NetAdapterRsc -Name $AdapterName |
    Format-Table -Property Name, IPv4Enabled, IPv4OperationalState, IPv4FailureReason -AutoSize

Write-Host "== Jumbo packet"
Get-NetAdapterAdvancedProperty -Name $AdapterName -RegistryKeyword "*JumboPacket" -ErrorAction SilentlyContinue |
    Format-Table -Property Name, DisplayValue, @{ Name = "RegistryValue"; Expression = { $_.RegistryValue -join "," } } -AutoSize

Write-Host "== Global offloads"
Get-NetOffloadGlobalSetting |
    Format-List -Property ReceiveSideScaling, ReceiveSegmentCoalescing, Chimney, TaskOffload

Write-Host "== TCP templates"
Get-NetTCPSetting |
    Where-Object { $_.AutoTuningLevelLocal } |
    Format-Table -Property SettingName, AutoTuningLevelLocal, CongestionProvider -AutoSize

Write-Host "== Legacy registry values (ignored by Windows)"
$tcpipParams = Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters"
$legacyFound = $false

foreach ($legacy in @("TcpWindowSize", "NumTcbTablePartitions", "MaxHashTableSize")) {
    if ($null -ne $tcpipParams.$legacy) {
        Write-Warning "$legacy = $($tcpipParams.$legacy) (ignored; safe to remove)"
        $legacyFound = $true
    }
}

if (-not $legacyFound) {
    Write-Host "None found."
}

Write-Host "== SMB Multichannel"
Get-SmbClientConfiguration | Format-List -Property EnableMultiChannel
Get-SmbMultichannelConnection -ErrorAction SilentlyContinue | Format-Table -AutoSize

Write-Host "== TCPv4 retransmits over $SampleSeconds seconds"
$counters = "\TCPv4\Segments Sent/sec", "\TCPv4\Segments Retransmitted/sec"
$samples = Get-Counter -Counter $counters -SampleInterval 1 -MaxSamples $SampleSeconds

$sent = ($samples.CounterSamples | Where-Object { $_.Path -like "*segments sent/sec" } | Measure-Object -Property CookedValue -Sum).Sum
$retrans = ($samples.CounterSamples | Where-Object { $_.Path -like "*segments retransmitted/sec" } | Measure-Object -Property CookedValue -Sum).Sum

if ($sent -gt 0) {
    "Retransmitted {0:N0} of {1:N0} segments ({2:P3})" -f $retrans, $sent, ($retrans / $sent)
} else {
    "No TCPv4 segments sent during the sample."
}

Run it on both hosts, save the output with the change ticket, and run it again after each change. The one-setting-at-a-time version of the fixes (RSS, RSC, checksum and LSO on, jumbo frames, autotuning back to Normal) is packaged as PowerShell: Windows Server – Tune the TCP Stack for 10GbE Workloads, which also warns that the adapter cmdlets bounce the NIC unless you pass -NoRestart.

None of this replaces the baseline

If I had to reduce this to an order of operations: measure the raw link with NTTTCP, confirm RSS is spreading load and the storage protocol is using more than one connection, confirm autotuning is Normal and delete any 2003-era registry values, check that RSC and the offloads are actually operating, then do jumbo frames last and prove them with a Don't Fragment ping. Each step is cheap to check and hard to argue with, which is exactly what you want at 2 a.m. when the storage team is sure it's the network and the network team is sure it's storage.

References