AI’s RAM Dilemma: Cloud Over Local
If you’ve tried to buy new electronics recently, from gaming consoles to iPhones, you’ll have dealt with prices that are hundreds higher than they were even a year ago. That’s because the price of RAM keeps rising as AI companies race to build datacenters for frontier LLM models.
OpenAI’s Sam Altman disagrees, saying many want local inference. “A lot of people want local inference,” the executive whose company is expected to go public sometime next year recently told the Cheeky Pint podcast.
The AI community is split between running lightweight open‑source models on phones and laptops versus using massive cloud datacenters controlled by tech giants. Altman backs the datacenter side, meaning the firms buying up all available memory and driving prices up for years.
“But from a global allocation perspective, actually, it’s quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used five percent of the time,” Altman continued. “You want maximum utilization in the cloud if you’re trying to deal with this very supply constrained environment. And somehow everyone’s intuition is the opposite of that.”
These “all your RAM are belong to us” proclamations were delivered in-between sips of a slowly warming IPA (all the tech CEOs are now forced to discuss world historical moments while awkwardly smashing beers on podcasts), but will have far-reaching consequences if they actually come to pass. It’s not hard to imagine a future where everything is rented, not just movies and Google spreadsheets but also the AI models these companies hope will micromanage and ultimately monetize every spare moment of your life.
Being able to play Grand Theft Auto 6 locally once it gets ported to PC is a waste in the mind of Altman. Stream it from the cloud in-between datacenters burning tokens to scrape the web and replace it with AI-generated overviews instead. Better yet, have ChatGPT agents bid on your behalf against other agents for how much the spare compute is worth it to you to take a nightly drive around Vice City. Please clap.