Memory Is the New Compute

Why the AI bottleneck moved from GPUs to bits — and what it means for a portfolio

Part one of two. Part two, "Mapping the Memory Supply Chain," walks through the companies at each link.

In August 2026, Peter Diamandis floated an idea that spread quickly: in the agentic era, what limits progress is no longer computing capacity but memory. Elon Musk publicly agreed.

The social media exchange is the least interesting part. What matters is that Musk had made the same argument, with numbers, in a far more demanding forum. On August 4, on SpaceX's second-quarter earnings call, responding to a question from Doug Anmuth of JP Morgan about supply-demand dynamics, he was explicit:

"The limiting factor currently is memory. The memory output is increasing by around 20% per year. Now, normally, that would be fantastically fast and amazing for any large mature industry. But ask yourself, is the demand increasing by 20% a year? No, the demand is increasing by 200% a year, maybe higher."

Coming from one of the world's largest buyers of AI infrastructure — Tesla and SpaceX build their own data centers — this is not rhetoric. It is a buyer's observation.

This article examines whether that observation supports an investment thesis, how much of it is already in prices, and which counterarguments a serious investor should be able to answer before acting.

1. What "memory" actually means in an agent

To see why this is not simply "more chip demand," you have to recognize that an agent's memory is not one thing. It is three layers, with speeds, capacities and costs that differ by orders of magnitude.

Language models are stateless: every call starts from zero and discards its context when it finishes. A chatbot lives comfortably with that limitation. An autonomous agent — a system that plans, executes, uses tools and maintains state over hours or days — does not. Sustaining that state requires all three layers at once:

KV-cache in HBM. When a model generates text token by token, it stores two matrices (keys and values) so it does not have to recompute the entire context at every step. That cache lives in the HBM soldered to the accelerator, with several TB/s of bandwidth and roughly 80 to 288 GB of capacity per GPU depending on the generation. The problem: the KV-cache grows linearly with context length and with the number of agent steps. An agent running 30 or 50 reasoning steps plus tool history can saturate an entire GPU's HBM. When that happens, either the agent "forgets" or parts of the cache get evicted, with the corresponding performance loss.

Working DRAM. Conventional server memory: hundreds of GB/s against HBM's several TB/s, but far larger and cheaper. It holds the active session state, allows KV-cache offload when HBM fills up, and supports multiple concurrent agents.

Persistent storage. SSDs, NAND, hard drives and the databases — vector or otherwise — where whatever the agent must remember between sessions actually lives. It is not loaded wholesale: the agent retrieves only what is relevant and promotes it to the upper layers.

What makes this relevant to an investor is that each layer has a distinct set of manufacturers:

| Layer | Speed | Capacity | Who produces it | |---|---|---|---| | KV-cache in HBM | Several TB/s | ~80–288 GB per GPU | SK hynix, Micron, Samsung; TSMC (base die); BESI, Hanmi, ASMPT (stacking) | | Working DRAM | Hundreds of GB/s | Terabytes per server | The same three; Nanya and Winbond in legacy DRAM; Rambus and Montage in the interface | | Persistent storage | Milliseconds | Effectively unlimited | SanDisk, Kioxia (NAND); Seagate, Western Digital (HDD); the software layer |

And the hierarchy is not merely under strain — it is reorganizing. SOCAMM2 and CXL expand the middle layer, and in August SanDisk and SK hynix published the first open specification for HBF (High Bandwidth Flash) through the Open Compute Project: a NAND-based tier between HBM and SSD, with up to 512 GB per stack and 0.4 to 3.0 TB/s.

One necessary clarification, because the slogan invites overstatement: it is not that compute is abundant. GPUs remain scarce and expensive. What has happened is that, at the margin and for agentic inference, the binding constraint shifted to the memory side. TrendForce describes the same phenomenon from the demand side: inference requests in agentic systems are evolving from single queries into continuous iterative cycles, with heavy users consuming up to four times more tokens than before.

The industrial consequence is twofold. First, HBM is the scarce link. Second — and less commented on — the wafer capacity allocated to HBM stops producing conventional DRAM, so the shortage propagates down the entire chain.

2. The numbers: a historic revision of expectations

In May 2026, TrendForce raised its global memory market forecast by an unusual magnitude:

| Segment | 2025 (actual) | 2026 (est.) | 2027 (proj.) | |---|---|---|---| | Total memory market | $224.7bn | $889.3bn | >$1.28tn | | DRAM | — | $618.7bn (+303% YoY) | $903.3bn (+46%) | | NAND Flash | — | $270.6bn (+281% YoY) | $379.4bn (+40%) |

The 2026 estimate was raised from $551.6bn in the prior report — a revision of more than 60% in a single quarter. The 2027 figure, from $842.7bn.

These numbers deserve careful reading, because they invite a misinterpretation. DRAM growth of 303% does not mean four times as many bits are being sold. It mainly means prices multiplied. Micron's bit shipments in its fiscal third quarter of 2026 rose by a low single-digit percentage; DRAM revenue rose 343% year over year. Virtually the entire gap is price.

This distinction is not a technicality. Price cycles revert. Volume cycles take longer.

3. What company results show

Corporate figures confirm the shortage with less ambiguity than sector projections.

Micron (MU), fiscal third quarter 2026 (reported June 24):

• Revenue: $41.5bn, +346% year over year, +74% sequentially.

• Gross margin: 84.9%, a company record, up from 39.0% a year earlier.

• Operating income: $33.7bn.

• Fourth-quarter guidance: $50bn in revenue and roughly 86% gross margin.

• 16 strategic customer agreements (take-or-pay) representing approximately $100bn in minimum contracted revenue and $22bn in upfront customer cash.

That last point deserves attention. Take-or-pay contracts with floor prices shift part of the cycle risk to the customer and fund capacity expansion in advance. That is a change in business structure, not just a good quarter.

SK hynix (SKHY) leads HBM with roughly 57-62% of the market depending on source and quarter (IDC estimated 63.2% by revenue in 2025). On July 10, 2026 it placed 177.9 million ADS at $149 on Nasdaq, raising approximately $26.5bn — the largest US listing ever by a foreign company, exceeding Alibaba's 2014 debut. Management maintains that the global memory shortage will extend beyond 2030.

Samsung Electronics is the third relevant player, recovering ground in HBM4 after yield problems in earlier generations, with additional NAND exposure.

All three manufacturers' HBM capacity is sold out for 2026. Micron has indicated that additional capacity will not be available until mid-2027 and that a new plant will not be operational until 2028.

4. Terafab: the buyer that wants to stop buying

There is one response to the shortage that deserves its own section, because it cuts both ways.

On August 6, Tesla and SpaceX confirmed they will build Terafab in Grimes County, Texas, about an hour northwest of Houston, with an initial investment of $16.8bn and at least 3,000 jobs. SpaceX's description is explicit on the point that matters here: a single facility combining logic, memory, packaging and testing under one roof, spanning more than 100 million square feet. Musk called it the largest and most valuable building on Earth. The full figures are larger: a county public hearing notice in May recorded $55bn for the first phase and up to $119bn for the complete buildout. Intel joined the project in April as a manufacturing partner with its 14A process — the first significant external commitment for its foundry business.

That memory falls within the project's scope is not an inference: Tesla has posted a Memory Process Integration Engineer role focused on end-to-end DRAM process integration and sub-20nm DRAM node development. It is the most concrete signal that vertical integration is not limited to logic.

The rest of the silicon program fills in the context: AI5 taped out on April 15, 2026 — fabricated by TSMC in Arizona and Samsung in Taylor, Texas — with volume production targeted for mid-2027, while AI6 and Dojo 3 advance in parallel.

For this thesis, Terafab means two opposite things, and both matter.

In the near term it confirms the diagnosis. When one of the world's largest buyers of AI infrastructure decides to spend tens of billions to avoid depending on memory supply, it is expressing with capital exactly what Musk said in words on the August 4 call. It is the most expensive possible validation that the shortage is perceived as structural rather than transitory.

In the long term it is a threat to the pillar of the thesis. If the captive-fab model works, the largest customers of Micron, SK hynix and Samsung stop being customers. Gross margins of 85% are precisely the incentive that makes that departure rational. Vertical integration is the natural response of a buyer facing a supplier with pricing power, and it usually arrives late but it arrives.

The timeline and probability deserve calibration, however. SpaceX's own IPO prospectus, filed in May, describes Terafab as a "general framework" with no binding commitments, leaving either party free to walk away. Tesla's track record on silicon schedules does not invite optimism either: AI5 was announced as "finished" in mid-2025, "almost done" in January 2026, and taped out in April — roughly two years behind original guidance.

Meanwhile, the dependence continues. In the AI5 tape-out photograph Musk posted, twelve SK hynix LPDDR5X modules are visible around the main die. On Tesla's late-July earnings call, Musk noted that Micron has been "making room for Tesla in the years to come." The world's most aggressive vertical integrator remains, today, a customer of the incumbents.

5. The counterarguments the thesis must answer

This is where a well-built thesis separates from a well-told narrative. There are six serious objections.

a) This is a price cycle, and price cycles revert. Gross margins of 85% in a historically cyclical industry are not a steady state; they are the peak of an imbalance. Micron, SK hynix and Samsung are all adding capacity, as are CXMT and the Taiwanese producers. If those additions land together in 2028-2029 — the classic semiconductor pattern — the correction can be severe. Micron's take-or-pay contracts mitigate this partially, not entirely.

b) Algorithmic risk is real and asymmetric. The whole thesis rests on the KV-cache growing linearly with context. If advances in compression, quantization or sparse attention materially reduce the bits required per token, structural demand contracts without warning. Work is already underway in that direction. This is the risk least likely to appear in manufacturers' presentations, for obvious reasons.

c) The market has already moved, and violently. Micron is up roughly 205-215% year to date in 2026, but fell 29% in July — its worst month since June 2005 — and trades approximately 25% below its June highs. SK hynix plunged 15.4% in Seoul on July 14, its worst day in nearly two decades, dragging the Korean index down 9% and forcing a trading halt. This is not an asset that can be sized like a defensive position.

d) Agentic adoption is still unproven. Gartner projects that more than 40% of agentic AI projects will be canceled by the end of 2027. It is important to be precise about the reason: Gartner cites escalating costs, unclear business value and inadequate risk controls — not memory limitations. Gartner further estimates that of the thousands of vendors presenting themselves as agentic, only around 130 genuinely are. If enterprise adoption disappoints, agentic inference demand moderates and the thesis with it.

e) Competition and geopolitics. The entry of CXMT and other Chinese producers into conventional DRAM, plus the concentration of the supply chain in Korea and Taiwan, introduce risks that fundamental analysis does not resolve.

f) Customers are building the exit. Terafab is the visible case, but the logic is general: 85% margins finance the vertical integration of the people paying them. This is a late-decade risk rather than a two-year one — and with execution probability clearly below one — but it is the kind of risk that erodes multiples before it erodes earnings.

In the other direction, and to avoid loading the argument one way: Deloitte projects that supply tightness and high prices will persist through 2029-2030, and Bank of America has characterized Micron's recent selloff as a buying opportunity. The gap between these views and the ones above is precisely where the potential return comes from.

6. How to think about sizing

If the thesis persuades, implementation matters more than conviction.

First, this is a cyclical position with a secular narrative, not the reverse. Structural demand is probably real; current margins almost certainly are not. Size it the way you size a semiconductor, not the way you size a utility.

Second, realized volatility is the relevant data point, not the thesis. An asset that falls 29% in a month within a year it rises 200% demands a position size compatible with that amplitude, and a real — not declared — tolerance for watching it.

Third, the horizon has to span the capacity cycle. New plants come online in 2027-2028. An 18-36 month thesis passes directly through that inflection point; it is worth having a view on what you will do when it arrives.

Fourth, there is overlap with what you already own. Anyone with exposure to semiconductor indices, to Nvidia, or to the large hyperscalers is already indirectly long this thesis. The question is not "should I take exposure?" but "how much more than I already have?"

Conclusion

The Diamandis and Musk observation is essentially correct: in the agentic era, holding state is as expensive as reasoning, and the industry spent a decade building infrastructure optimized for the latter. That mismatch is real, structural, and still correcting.

But a correct observation does not automatically produce an attractive trade. The market has already repriced memory to historic levels; manufacturers' margins sit at levels no semiconductor industry has sustained; and the stocks already reflect much of the story, with volatility that demonstrates it.

The defensible version of the thesis is not "memory is the new compute, buy memory." It is more modest and more useful: memory has stopped being a cost component and become a strategic asset, and that change in status will probably outlive the price cycle that made it visible. Anyone investing in that idea should be prepared to sit through the margin reversion that arrives when capacity catches demand — and to distinguish, in the meantime, between the part of their return that comes from structural change and the part that comes simply from prices going up.

Part two of this series, "Mapping the Memory Supply Chain," walks link by link through the companies producing each layer — from NAND and hard drive makers to HBM stacking equipment and interface chips — and explains which genuinely belong to this thesis and which only resemble it.

This content is for informational and educational purposes only. It does not constitute investment advice, an offer, or a recommendation to buy or sell any security, and it does not take into account the objectives, financial situation or particular needs of any investor. Investing in semiconductor equities involves significant risks, including possible loss of principal, and volatility substantially above that of the broad market. References to specific companies are illustrative and do not constitute recommendations. Past performance does not guarantee future results.

Sources

• SpaceX — Q2 2026 earnings call (August 4, 2026), Elon Musk's remarks on memory as the limiting factor

• TrendForce — Agentic AI Drives Structural Expansion in Memory Demand (May 29, 2026)

• TrendForce — Diverging Memory Market Outlook in 2027 (July 30, 2026)

• Micron Technology — Fiscal Q3 2026 results and prepared remarks (June 24, 2026)

• SK hynix — SEC Form DRS/A and coverage of the Nasdaq ADS offering (July 10, 2026)

• SpaceX / Tesla — Terafab announcement (August 6, 2026), Grimes County public hearing notice (May 2026), and SpaceX IPO prospectus

• Reuters, TechCrunch, CNBC, Electrek — Coverage of Terafab, Intel's participation, and the AI5 tape-out

• Gartner — Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (June 25, 2025)

• Counterpoint Research / IDC — Global DRAM and HBM market share

• Deloitte — Memory supply tightness projections through 2029-2030