SK hynix pitched a shared memory pool that cut AI first-token latency 81% in its tests

On November 12 at Intel's▼−4.19% — Intel, down 4.19 percent today AI Summit in Seoul, SK hynix▼−5.27% — SK hynix, down 5.27 percent today vice president Joo Young-pyo pitched a shared memory pool: racks of DRAM, the standard memory inside every server, that several GPU and CPU servers reach over CXL and RDMA links instead of each carrying its own high-bandwidth memory, or HBM ¹.

In the company's internal tests, average time to first token, the wait before a model starts answering, fell from over 80 seconds to about 15 ¹. SK hynix is pitching the remedy for the shortage that makes it richest.

One cache grows past what any processor package holds

Agentic software runs multi-step tasks across long conversations, and every turn adds to the key-value (KV) cache, the working memory a model keeps across a conversation ¹². Once that cache outgrows the HBM beside the processor, systems evict it and recompute, and the user waits ¹.

SK hynix's pool keeps one copy that many servers read over CXL (Compute Express Link, which lets a processor reach memory outside its own package) and RDMA, which lets machines read one another's memory without routing requests through a processor ¹².

At October's Open Compute Project summit the company showed the rack version, its Memory-centric AI Rack System (MARS) ². Solidigm, the solid-state drive maker SK hynix owns, and Metrum AI took a different route in a joint demo, holding the same cache on cheap solid-state drives and cutting energy per GPU request 26.5-fold ²³.

A sold-out, dear book sets the price the pool undercuts

The pitch competes with the product. SK hynix told investors on its third-quarter earnings call in late October↗ that its DRAM and HBM volumes are effectively sold out through 2029 ³⁴, covered by long-term agreements with 14 major customers ⁵.

One customer alone accounted for 13.35% of first-half revenue↗; Korean media name it Nvidia▼−3.21% — Nvidia, down 3.21 percent today, which SK hynix has not confirmed ⁵. Daishin Securities analysts estimate SK hynix leads Nvidia's supply of HBM4, the newest generation ⁵.

Every one of those contracts prices memory as scarce. The OCP presenters were pointing the other way: their shared destination is disaggregated, tiered memory, layers running from fastest and priciest to slower and cheaper ². That ordering puts a shared DDR5 pool, slower than the HBM on the package, below it on cost ².

New accelerator designs pull memory toward the cheaper tiers

Qualcomm▼−3.52% — Qualcomm, down 3.52 percent today, a chip designer, stacks LPDDR (low-power DRAM) directly on the compute die in its HBC design, with the Dragonfly AI250 accelerator due to sample in the first half of 2027 ⁶.

Nvidia has asked Samsung, SK hynix and Micron▼−4.50% — Micron, down 4.50 percent today to build base dies for NVHBM, the custom memory it co-designs with them, on the 3nm process of TSMC▼−2.68% — TSMC, down 2.68 percent today, a contract chip foundry ⁶. If agentic inference settles into shared pools and stacked LPDDR, some of those bytes leave HBM's price umbrella.

Both halves of the bet sit in one share

SK hynix holds an HBM franchise priced on Nvidia's roadmap and on scarcity, and a DRAM, CXL and rack-scale franchise that grows when scarcity ends. The tension broke into the share price on November 13, when the stock fell 3%↗ after Nvidia co-founder and chief executive Jensen Huang resigned and foreign investors turned sellers ⁵⁷.

Samsung Electronics▼−0.96% — Samsung Electronics, down 0.96 percent today is positioned to gain if the scarce half fades ⁵. SK hynix has been building toward the abundant half since October, when it showed a CXL memory product built on startup XCENA's chip, as we reported at the time ⁸.

The design is still a demonstration, with three tests ahead

None of this is deployed. The pooling figures are internal tests, and the rack designs await validation with Intel, customers and partners from the system-design phase onward ¹². Memory spread across CXL pools, DRAM and solid-state drives pays off only when the stack knows what data belongs where and when to move it ².

Watch three results: whether that validation reproduces the latency figures, whether the sold-out agreements hold their pricing as pooled DRAM ships, and whether Samsung's HBM4 delivers the 2027 revenue lead Counterpoint Research expects it to take from SK hynix ¹⁵. Each one lands on the same SK hynix share, where both halves of the thesis are priced together.