One CPU SKU Is Not One Latency Distribution
006 and 007 showed that nominally identical GPUs differ, and that a synchronous cohort pays for its slowest member. This note carries the argument to CPUs serving fan-out requests, where the amplifier is different: not the maximum over a cohort, but queueing near saturation. A request that needs every shard misses its deadline if any shard is late, and how late a shard runs depends on its slack — service rate minus arrival rate — times the time remaining.
Electrical spread is measured; capacity spread is conditional
In a June 2026 preprint, Crop, Moore and Pasricha measured ten Xeon 6314 (Ice Lake) parts of the same SKU at 2.0 GHz and an average die temperature of 50°C, and found core voltage varying by up to 40 mV, leakage differing by nearly 2×, and dynamic capacitance by up to 13%.1 That establishes electrical heterogeneity. Measured at a fixed frequency, it is a difference in power, not throughput.
It becomes a capacity difference mainly when a power or thermal limit binds. Inadomi et al. show the conversion directly.2 Without a cap, 2,386 Sandy Bridge processors showed almost no performance variation despite up to 23% variation in power. Under RAPL caps on a 1,920-module Ivy Bridge system, cutting the module budget for DGEMM from 110 W to 70 W widened the CPU frequency spread from 1.20× to 1.40×, and module power variation produced up to 64% performance variation across application ranks. The conversion is not automatic: for MHD, a 60 W cap produced a 1.75× frequency spread while per-rank execution time stayed within about 1%, because, as the authors put it, CPU-boundedness, memory and synchronization characteristics decide the impact. And “mainly” is not “only”: one uncapped system of 64 processors showed 17% performance variation alongside 21% in power. All of this is 2013-era HPC under deliberate caps, not stock-limit cloud servers — evidence for the mechanism, not a calibration of how slow a production host runs.
The queueing amplifier
Model each shard as an independent M/M/1 queue — Poisson arrivals, exponential service, first-come first-served, a single-server abstraction of a many-core host. A fan-out request that must wait for all N shards misses its remaining deadline D with probability
Take 100 shards at 900 req/s each and 100 ms of remaining deadline. With every shard at 1000 req/s, the request miss rate is 0.45%. Now make one shard run at 920 req/s — an 8% shortfall chosen purely for illustration, not measured on any part and unrelated to the 8% GPU statistic 006 discussed. The miss rate jumps to 13.9%. Whatever the dependence between shards, if that host’s own queue behaves as M/M/1, it alone imposes a 13.5% floor. Against a 1% budget, any single-shard shortfall above about 4.8% breaches it at this load — and the load matters as much as the shortfall:
| Load per shard | All healthy | One shard at 920 req/s |
|---|---|---|
| 850 req/s | 0.00% | 0.09% |
| 880 req/s | 0.06% | 1.89% |
| 900 req/s | 0.45% | 13.92% |
| 910 req/s | 1.23% | 37.56% |
Headroom or placement
There are two ways to buy the budget back. Cut load fleet-wide to about 874 req/s — 2.9% of capacity — and the miss rate returns to 1%. Or give the weak host only what it can carry: sizing its shard to 92% of the load restores 0.46% at a cost of 0.08% of aggregate capacity. Headroom looks affordable here only because the worst host is known. Across a fleet nobody has measured, headroom has to be sized for a deficit nobody has observed; placement needs the measurement, but then costs almost nothing.
None of the tools is new. Dean and Barroso prescribed micro-partitions and latency-induced probation for slow machines,3 and PAL places GPU work by measured, per-application variability.4 Hedged requests and retries remain right for transient stragglers; repeatedly hedging a persistently weak host becomes an operating subsidy. What is missing is upstream: CPU procurement acceptance that measures service curves across supported power limits and hands schedulers capability classes before a host takes traffic. The qualification sweep itself is the one 011 describes; the measured curve is 008’s workload-defined service unit at host scale.
The diagnostic
The question to ask of each host is counterfactual: how would the same request stream perform at matched temperature, power limit, NUMA placement and software state? Join queue wait and service time with effective frequency (APERF/MPERF counters), package energy, throttle reasons where the platform exposes them, memory traffic and workload identity, and use controlled replay and component swaps to separate persistent silicon effects from cooling, firmware and interference. A tenant who cannot see the hardware has a cruder version of the same tool: benchmark at launch, and replace hosts that come up slow.
References
- Crop, Moore, Pasricha, “Revisiting ‘Cooler is Better’: ITD-Aware Per-CPU Thermal Optimization for Sustainable Data Center Operation,” preprint, §3. arXiv:2606.11163
- Inadomi et al., “Analyzing and Mitigating the Impact of Manufacturing Variability in Power-Constrained Supercomputing,” SC ’15. doi:10.1145/2807591.2807638
- Dean, Barroso, “The Tail at Scale,” Communications of the ACM 56(2), 2013. research.google
- Jain, Tran, Chen, Sinclair, Venkataraman, “PAL: A Variability-Aware Policy for Scheduling ML Workloads in GPU Clusters,” SC ’24. arXiv:2408.11919
Provenance note, per house style. Quoted: the Xeon 6314 voltage, leakage and capacitance spreads (ref 1); the uncapped and capped power, frequency and performance variation figures (ref 2). Illustrative: 100 shards, 900 and 1000 req/s, the 920 req/s weak shard, 100 ms of deadline and the 1% budget. Computed on this page from them: every miss rate, the 13.5% single-shard floor, the 4.8% breach threshold, the slack rule, the sensitivity table, and both remedies.