---
title: "SK hynix pitched a shared memory pool that cut AI first-token latency 81% in its tests"
description: "The memory maker whose HBM is sold out through 2029 is now pitching cheaper pooled DRAM as the answer to the AI memory crunch. If the design catches on, some of the scarcity premium would leave its own product, and both franchises sit in one share."
publisher: "The Inference"
section: "Chips"
published: 2026-11-14T16:12:16.387Z
modified: 2026-11-14T16:12:16.387Z
canonical: https://theinference.org/article/sk-hynix-pitched-a-shared-memory-pool-that-cut-ai-first-token-latency-81-in-its-tests
language: en
keywords: "SK hynix, HBM, Memory, Nvidia, Semiconductors"
---

# SK hynix pitched a shared memory pool that cut AI first-token latency 81% in its tests

> The memory maker whose HBM is sold out through 2029 is now pitching cheaper pooled DRAM as the answer to the AI memory crunch. If the design catches on, some of the scarcity premium would leave its own product, and both franchises sit in one share.

On November 12 at [Intel](https://theinference.org/markets/companies/intel)'s AI Summit in Seoul, [SK hynix](https://theinference.org/markets/companies/sk-hynix) vice president Joo Young-pyo pitched a shared memory pool: racks of DRAM, the standard memory inside every server, that several GPU and CPU servers reach over CXL and RDMA links instead of each carrying its own high-bandwidth memory, or HBM [1].

In the company's internal tests, average time to first token, the wait before a model starts answering, fell from over 80 seconds to about 15 [1]. SK hynix is pitching the remedy for the shortage that makes it richest.

## One cache grows past what any processor package holds

Agentic software runs multi-step tasks across long conversations, and every turn adds to the key-value (KV) cache, the working memory a model keeps across a conversation [1][2]. Once that cache outgrows the HBM beside the processor, systems evict it and recompute, and the user waits [1].

SK hynix's pool keeps one copy that many servers read over CXL (Compute Express Link, which lets a processor reach memory outside its own package) and RDMA, which lets machines read one another's memory without routing requests through a processor [1][2].

At October's Open Compute Project summit the company showed the rack version, its Memory-centric AI Rack System (MARS) [2]. Solidigm, the solid-state drive maker SK hynix owns, and Metrum AI took a different route in a joint demo, holding the same cache on cheap solid-state drives and cutting energy per GPU request 26.5-fold [2][3].

## A sold-out, dear book sets the price the pool undercuts

The pitch competes with the product. SK hynix told investors on [its third-quarter earnings call in late October](https://biz.heraldcorp.com/article/10917362) that its DRAM and HBM volumes are effectively sold out through 2029 [3][4], covered by long-term agreements with 14 major customers [5].

One customer alone accounted for [13.35% of first-half revenue](https://www.koreajoongangdaily.com/business/jensen-huangs-exit-clouds-future-nvidia-demand-for-korean-chipmakers/12970507); Korean media name it [Nvidia](https://theinference.org/markets/companies/nvidia), which SK hynix has not confirmed [5]. Daishin Securities analysts estimate SK hynix leads Nvidia's supply of HBM4, the newest generation [5].

*Chart: **Estimated 2026 share of Nvidia's HBM4 supply** Bar chart of estimated 2026 HBM4 supply shares to Nvidia: SK hynix leads, followed by Samsung and Micron.*

*Estimated share of Nvidia's HBM4 supply in 2026, per Daishin Securities; SK hynix has not confirmed Nvidia is its largest customer. Source: Korea JoongAng Daily, November 12, 2026. [5]*

Every one of those contracts prices memory as scarce. The OCP presenters were pointing the other way: their shared destination is disaggregated, tiered memory, layers running from fastest and priciest to slower and cheaper [2]. That ordering puts a shared DDR5 pool, slower than the HBM on the package, below it on cost [2].

## New accelerator designs pull memory toward the cheaper tiers

[Qualcomm](https://theinference.org/markets/companies/qualcomm), a chip designer, stacks LPDDR (low-power DRAM) directly on the compute die in its HBC design, with the Dragonfly AI250 accelerator due to sample in the first half of 2027 [6].

Nvidia has asked Samsung, SK hynix and [Micron](https://theinference.org/markets/companies/micron) to build base dies for NVHBM, the custom memory it co-designs with them, on the 3nm process of [TSMC](https://theinference.org/markets/companies/tsmc), a contract chip foundry [6]. If agentic inference settles into shared pools and stacked LPDDR, some of those bytes leave HBM's price umbrella.

## Both halves of the bet sit in one share

SK hynix holds an HBM franchise priced on Nvidia's roadmap and on scarcity, and a DRAM, CXL and rack-scale franchise that grows when scarcity ends. The tension broke into the share price on November 13, when [the stock fell 3%](https://financefeeds.com/sk-hynix-stock-falls-3-to-1-216-million-won-as-kospi-gives-up-5504-intraday-high/) after Nvidia co-founder and chief executive [Jensen Huang](https://theinference.org/markets/people/jensen-huang) resigned and foreign investors turned sellers [5][7].

[Samsung Electronics](https://theinference.org/markets/companies/samsung) is positioned to gain if the scarce half fades [5]. SK hynix has been building toward the abundant half since October, when it showed a CXL memory product built on startup XCENA's chip, [as we reported](https://theinference.org/article/marvell-trades-at-77-4-times-earnings-as-sk-hynix-s-first-commercial-memory-runs-on-a-startup-s-chip) at the time [8].

## The design is still a demonstration, with three tests ahead

None of this is deployed. The pooling figures are internal tests, and the rack designs await validation with Intel, customers and partners from the system-design phase onward [1][2]. Memory spread across CXL pools, DRAM and solid-state drives pays off only when the stack knows what data belongs where and when to move it [2].

Watch three results: whether that validation reproduces the latency figures, whether the sold-out agreements hold their pricing as pooled DRAM ships, and whether Samsung's HBM4 delivers the 2027 revenue lead Counterpoint Research expects it to take from SK hynix [1][5]. Each one lands on the same SK hynix share, where both halves of the thesis are priced together.

## Takeaway

SK hynix is pitching a shared DRAM pool that undercuts the scarcity pricing of its own sold-out HBM book, so whichever design wins, the same share carries both bets.

## Sources

1. [Crypto Briefing, SK Hynix and Intel explore shared memory pools to ease AI bottlenecks, 13 November 2026](https://cryptobriefing.com/sk-hynix-intel-shared-memory-pool-ai/)
2. [Crypto Briefing, SK hynix, Solidigm and others pitch AI memory beyond HBM at OCP Global Summit, 13 November 2026](https://cryptobriefing.com/sk-hynix-solidigm-ai-memory-ocp-summit/)
3. [The Herald Business, SK hynix CEO says DRAM, HBM sold out through 2029, vows shareholder returns, 28 October 2026](https://biz.heraldcorp.com/article/10917362)
4. [Chosunbiz, SK hynix sells out DRAM and HBM through 2029 on LTAs, 28 October 2026](https://biz.chosun.com/en/en-it/2026/10/28/K6KPE7VGMNFFZIUWF3F5U3PKEA/)
5. [Korea JoongAng Daily, Jensen Huang's exit clouds future Nvidia demand for Korean chipmakers, 12 November 2026](https://www.koreajoongangdaily.com/business/jensen-huangs-exit-clouds-future-nvidia-demand-for-korean-chipmakers/12970507)
6. [TrendForce, TSMC Pushes Back on 3D DRAM Rumors: Three Memory Plays Where It Still Has a Say, 13 November 2026](https://www.trendforce.com/news/2026/11/13/news-tsmc-pushes-back-on-3d-dram-rumors-three-memory-plays-where-it-still-has-a-say/)
7. [FinanceFeeds, SK Hynix Stock Falls 3% to 1.216 Million Won as Kospi Gives Up 5,504 Intraday High, 13 November 2026](https://financefeeds.com/sk-hynix-stock-falls-3-to-1-216-million-won-as-kospi-gives-up-5504-intraday-high/)
8. [Tech Times (via MSN), SK Hynix debuts CXL computational memory on XCENA MX1; Samsung commits as foundry, October 2026](https://www.msn.com/en-us/technology/tech-companies/sk-hynix-debuts-cxl-computational-memory-on-xcena-mx1-samsung-commits-as-foundry/ar-AA2fy8fm)


---

SK hynix pitched a shared memory pool that cut AI first-token latency 81% in its tests — The Inference. Canonical: https://theinference.org/article/sk-hynix-pitched-a-shared-memory-pool-that-cut-ai-first-token-latency-81-in-its-tests