Google’s TurboQuant Shrinks AI Memory Sixfold as Data Centers Stay Power-Bound

Nvidia told investors this month that its next rack-scale chip platform will ship more than a year late. The stock barely moved.

Google published a compression algorithm called TurboQuant back in March, claiming it could shrink the memory large language models need by at least six times. Memory chip stocks fell within hours of the announcement. Four months later, the constraint holding back America’s AI data center boom turned out to be electricity and permits, not chips or memory at all.

Inside Google’s TurboQuant Algorithm

Every time a large language model answers a prompt, it keeps a running memory of everything it has processed so far. Engineers call this the key-value (KV) cache, and it grows with every additional token in a conversation. On long documents or sprawling codebases, that cache can outgrow the model itself.

Google Research scientists Amir Zandieh and Vahab Mirrokni built TurboQuant to shrink that cache without losing accuracy, working with collaborators from New York University, Google DeepMind and South Korea’s KAIST. The work builds on two earlier papers from the same group, one on the QJL method presented at AAAI 2025 and another on PolarQuant due at AISTATS 2026, part of a research arc VentureBeat traced back to 2024.

The algorithm works in two steps. First, it randomly rotates a model’s data vectors, spreading out values that would otherwise choke standard compression. Second, it applies a tiny, one-bit correction that cleans up the rounding error the rotation leaves behind.

Google says the result is a cache compressed to as little as 3 bits per value, down from an industry standard of 16, for a memory reduction of at least six times with no measurable drop in accuracy. At 4-bit precision, the algorithm also computed attention up to eight times faster on Nvidia’s H100 chips than an uncompressed baseline.

This has potentially profound implications for all compression-reliant use cases, including and especially in the domains of search and AI.

Google Research wrote that in the blog post announcing the work, timed ahead of a formal presentation at the International Conference on Learning Representations (ICLR) in Brazil this year.

Independent testing since the announcement found gains more modest than Google’s own headline numbers, often four to five times rather than six. Researchers who ran the comparisons still called the results meaningfully good, just short of the paper’s best case.

What Made Memory Stocks Fall So Fast

Micron, Western Digital and SanDisk all dropped within a day of Google’s announcement, as traders bet that software compression would cut into demand for the physical chips that store AI’s working memory. Micron fell 3%, Western Digital lost 4.7% and SanDisk slid 5.7%.

Wells Fargo analyst Andrew Rocha said the algorithm directly attacks the cost curve for memory in AI systems. Rocha and other analysts also cautioned that overall demand for AI memory remains strong, and that compression tools have existed for years without denting how much hardware the industry actually buys.

Part of the SanDisk move may have had a separate cause. The company had announced plans to invest $1 billion in Taiwanese memory maker Nanya Technology the same week, muddying any clean read on how much of its drop belonged to TurboQuant.

Cloudflare CEO Matthew Prince compared the moment to DeepSeek’s arrival in 2025, when a leaner Chinese AI model briefly rattled the same stocks.

TurboQuant Fixes Only Half of AI’s Memory Problem

TurboQuant only compresses memory during inference, the process of running an already-trained model to answer a query. It does nothing for training, the far larger, one-time process of building a model in the first place, which still demands enormous amounts of high-bandwidth memory.

Training runs have filled much of the memory industry’s order books over the past two years, far more than routine inference traffic has. The algorithm also has not shipped anywhere yet. Google’s own researchers describe it as a lab result rather than a deployed product.

Technology analyst Tim Bajarin drew a similar line in a Forbes column, arguing that DeepSeek’s savings reshaped assumptions about the cost of training models industry-wide, while TurboQuant is a narrower optimization confined to serving them afterward.

Power Is Holding Back America’s Data Centers

Nvidia CEO Jensen Huang called this moment the largest infrastructure buildout in history, with his company positioned as its chief supplier of chips. Memory chips were never the bottleneck behind delays in that buildout.

In its most recent quarterly filing with the U.S. Securities and Exchange Commission, Nvidia listed the availability of sufficient data center capacity or energy as a risk to its own revenue, warning that any shortage of these resources “could impact our future revenue and financial performance.”

Hyperscalers are on pace to spend roughly $1 trillion on data centers between 2024 and 2026, according to real estate firm JLL. At that scale, even small efficiency gains compound quickly, though the power grid was never built to match it.

Four pressures are colliding at once:

• Equipment bottlenecks – transformer and switchgear lead times have stretched toward three to four years in some cases, according to industry trackers.
• Labor shortages – construction executives working on data center projects for OpenAI and other developers cited a lack of electricians and pipe fitters.
• Grid queues – interconnection studies in major U.S. markets now routinely take up to four years before a new site can draw power.
• Blocked projects – Data Center Watch counted at least 75 U.S. data center projects worth roughly $130 billion blocked or delayed in the first quarter of 2026 alone.

The pattern repeats across nearly every fresh report on the sector this year, regardless of what happens to memory chip prices.

Nvidia Delayed Its Next Chip Platform Anyway

Nvidia’s own roadmap makes the point better than any critique of TurboQuant could. The company is delaying its Kyber rack-scale architecture by more than 12 months, pushing the launch of its next-generation platform out to 2028.

Shares barely reacted. Older platforms are still selling well enough to absorb the wait, and hyperscalers keep pouring money into existing high-density racks and the power infrastructure needed to run them.

Nvidia’s next earnings report will be the clearer test of whether compression breakthroughs like TurboQuant have started to change order books at all.

• 24.1% – Nvidia’s stock gain over the past year, even after news of the Kyber delay.
• 40% – share of AI data centers Gartner expects to be power-constrained by 2027.
• 24 to 36 months – typical grid capacity approval timeline in major U.S. and European markets.
• August 26 – date of Nvidia’s next earnings report, the next real test of demand.

Does AI Efficiency Ever Shrink Demand?

Rarely, if the history of computing is any guide. Cheaper storage has always led people to store more, and faster bandwidth gets consumed by heavier applications almost as soon as it arrives. Google’s own breakthrough may follow that same pattern instead of shrinking AI’s footprint.

Economists have a name for gains that expand consumption instead of shrinking it. Bajarin framed TurboQuant as part of a broader shift, with AI labs competing on efficiency and cost as much as raw model size, and cheaper inference making AI more accessible rather than simply cutting hardware budgets.

Nvidia’s shares kept climbing the same month the Kyber delay became public. The wall socket is still what stands between America’s AI ambitions and its next wave of data centers.

Frequently Asked Questions

What is TurboQuant?

TurboQuant is a compression algorithm from Google Research that shrinks the key-value cache large language models rely on during inference, cutting memory needs by at least six times with what Google describes as no loss in accuracy. It is set for a formal presentation at ICLR 2026 in Brazil.

What’s the difference between AI training memory and inference memory?

Training builds a model once, in a long and compute-heavy process. Inference runs that finished model for every user request, continuously, and now accounts for roughly 80% to 90% of total AI compute load across major vendors. TurboQuant only compresses the inference side, leaving training’s separate memory demands untouched.

How big is the memory problem TurboQuant is trying to solve?

Running a 70-billion-parameter Llama model with a 1-million-token context window can require roughly 328 gigabytes of VRAM just for its key-value cache, versus 140 gigabytes for the model’s own weights, according to Darshan Fofadiya, an AI researcher at Amazon. Compressed to 3.5 bits, that cache shrinks to about 72 gigabytes, small enough to fit on a single Nvidia H100 chip.

Is TurboQuant already being used anywhere?

Not in production systems yet. Google describes it as a research result rather than a shipped product, though independent developers began porting the technique into open-source libraries like MLX and llama.cpp within 24 hours of the announcement. One developer, posting as Prince_Canuma, tested it on the Qwen3.5-35B model almost immediately.

Why are data centers still delayed if AI needs less memory now?

Because memory chips were never the constraint holding up construction. Grid interconnection studies take up to four years in busy markets, and public opposition has grown quickly. An Ipsos survey found opposition to nearby data centers jumped from roughly half of Americans to 70% within months.

Could compression like this eventually let AI run entirely on phones?

That is one of the more plausible upsides. Smaller memory footprints make on-device AI more realistic, and Forbes analyst Tim Bajarin pointed to on-device applications as one area where TurboQuant’s memory savings could matter most, even though Google has not announced any phone deployment of the technology.