Huge Pages, Tiny Hotspots
Huge pages exchange placement granularity for translation reach: fewer translations mean fewer page walks and less pressure on the TLB, the cache of recent address translations. The trade was nearly free while every byte of a mapping had the same destination. NUMA was the first exception — Gaud et al. found large pages degraded 25–30% of the applications they tested, by 5% to 43%, through poor locality and imbalance1 — and tiered memory makes it the rule. Once a fast tier and a slow tier sit behind the same addresses, the best unit for translation and the best unit for migration can have very different costs.
The problem is not new. Thermostat, which migrates cold huge pages to slow memory, notes that placing the entirety of each 2 MB page “consumes extra fast memory space for 2MB pages with a small hot footprint,” and leaves placing only the hot portions “for future work.”2 Carrefour-LP and MEMTIS split hot huge pages and move only the hot part;1,3 Linux’s DAMON tracks access by region rather than by mapping.4 What this note adds is the price: the capacity a placement occupies is not free, and the cost of stranding cold bytes in the fast tier can be computed.
One hot region, one big page
Consider an illustrative 2 MiB page holding one repeatedly accessed 64 KiB region, and assume — as a hypothesis — that the hot region is contiguous, aligned, and the rest of the page untouched. That premise is workload-dependent: MEMTIS found hot huge pages with low subpage utilization in Silo, where migrating them “would waste memory in the fast tier,” but not in Liblinear.3 Where it holds, migrating the whole page moves 32 times the hot data and fills fast-tier capacity with 31/32 cold bytes. Moving only the hot region means splitting the mapping — on x86-64, from one 2 MiB entry to 4 KiB entries with 512× less reach each — and paying for it: MEMTIS calls a huge-page split “an expensive operation involving data copy and TLB shootdown.”3 Neither choice is correct in general.
Two screens, not one
The original form of this argument compared fine placement against doing nothing. The decision actually needs two tests. First, is moving worth it at all? Let A be expected future hot accesses, ΔL the exposed latency each saves, Mf the cost of migrating and remapping the hot region, and T the extra translation cost of the finer mapping over the interval — all in seconds of exposed stall:
Second, given a move, is fine placement better than moving the whole page? The hot accesses are served from the fast tier either way, so their saving cancels, and what remains is capacity against the cost of splitting:
Fine placement wins only when both hold. These are screening tests, not cycle-accurate predictions: memory-level parallelism, overlapping migration and queueing decide how much latency actually reaches execution, and ΔL can be zero or negative when the fast tier’s own bandwidth is saturated — itself a reason to leave data where it is.
Pricing the fast tier
The term existing placement policies leave implicit is λ. Meta’s Vistara deployment gives one cited point: attaching DDR4 recycled from decommissioned servers through CXL, with a relative cost per GB of 0.13 against local DDR5 at 1, about 10× less bandwidth (57.6 against 614.4 GB/s per configuration) and about 60% higher latency, roughly 150 ns of it from the expander datapath.5 That is a normalized, Meta-specific ratio built on memory the authors call “near zero-cost memory expansion through recycling” — not a market price for new CXL memory. Thermostat swept a hypothetical 0.2–0.33× for new slow media.2 Either way, each cold gigabyte left in the fast tier costs the difference. Per unit of hot data, whole-page placement strands 31 units of cold data, so its extra cost over fine placement is:
| Far-tier cost / fast-tier cost | Source | Extra cost per hot unit (fast-tier units) |
|---|---|---|
| 0.13 | Vistara (recycled DDR4 behind CXL) | 27.0 |
| 0.20 | Thermostat sweep, low end (hypothetical) | 24.8 |
| 0.33 | Thermostat sweep, high end (hypothetical) | 20.8 |
In other words, under the illustrative 32:1 page-to-hotspot ratio, whole-page placement makes the effective fast-tier price of each hot gigabyte up to 32× its list rate, of which 27–21 units are recoverable by placing finely. That is the term to weigh against the split’s data copy, TLB shootdowns and lost reach. It is 005’s “know the denominator” at page granularity: the price of the fast tier per byte placed is not its price per byte used.
Separate observation from translation
A placement controller should observe access frequency below the current migration unit where practical, keep enough history to judge stability, know whether a mapping can be split and later coalesced, and compare four options — whole-page migration, finer relocation, replication where semantics permit, and leaving the data alone. Remapping costs have to be visible to it. Movement on a short burst can otherwise spend bandwidth twice: once entering the fast tier, and again when another transient hotspot displaces it. The metric that settles it is work completed per dollar of fast-tier capacity, counting the traffic the policy itself generates — 008’s useful-work denominator applied to memory. A lower TLB miss rate can coexist with worse service if the chosen mappings stranded the wrong data near the processor.
References
- Gaud, Lepers, Decouchant, Funston, Fedorova et al., “Large Pages May Be Harmful on NUMA Systems,” USENIX ATC ’14.
- Agarwal, Wenisch, “Thermostat: Application-transparent Page Management for Two-tiered Main Memory,” ASPLOS ’17, §6. doi:10.1145/3037697.3037706
- Lee, Monga, Min, Eom, “Memtis: Efficient Memory Tiering with Dynamic Page Classification and Page Size Determination,” SOSP ’23. doi:10.1145/3600006.3613167
- Linux kernel documentation, DAMON design. docs.kernel.org
- Gholkar et al., “Vistara: Making CXL Real — Full Path from ASIC Design and OS Support to Hyperscale Deployment,” ISCA ’26, Table V. aisystemcodesign.github.io
Provenance note, per house style. Illustrative: the 2 MiB page and 64 KiB hot region, and the hypothesis that the region is contiguous and the rest of the page cold. Computed on this page: the 32× fan-out, the 512× reach loss, and the stranded-cost table. Quoted: the NUMA large-page results (ref 1); Thermostat’s whole-page placement, future-work statement and 0.2–0.33× sweep (ref 2); MEMTIS’s Silo/Liblinear finding and split cost (ref 3); Vistara’s relative cost, bandwidth and latency figures (ref 5).