Small Models, Big Deal: The Case for Tiny AI

The headline race in AI has always run in one direction: bigger. More parameters, more data, more datacenter. But the most consequential trend of the past two years has been running quietly the other way — models small enough to live on your laptop, your phone, eventually your thermostat, that punch absurdly above their weight.

The recipe isn't magic. Distill the big model's judgment into the small one's frame. Curate training data like a librarian instead of a vacuum cleaner. Specialize ruthlessly: the model that only has to be brilliant at one domain can be a fraction of the size of the one that has to be passable at everything.

The physics argument

The case for tiny AI is ultimately thermodynamic. Intelligence that requires a datacenter has a meter running on every thought; intelligence that fits in your pocket rounds to free. Every task that migrates from the big model to the small one is a task that stops needing the network, stops sharing your data, stops costing per token — and starts working on an airplane.

The most important spec sheet number of the next decade might be tokens per watt.

Privacy is the sleeper benefit. The assistant that processes your life entirely on your device changes the trust equation — not because anyone promises to behave, but because the data never leaves in the first place. Architecture beats policy.

The honest division of labor

None of this dethrones the giants. The frontier models remain the R&D lab of the whole ecosystem — the place capability is discovered before it's compressed. The emerging pattern is a division of labor: big models for the novel and the hard, small models for the routine and the private, with a router in between deciding which is which.

Which means the interesting question is no longer "how big can we build?" It's "how little intelligence does this task actually need?" — and the answer, far more often than the marketing would suggest, is: less than you think.