Model compression, better NPUs and new developer tools are bringing large language models onto devices — changing privacy, battery life and who gets paid.
Illustration by IMF Alpha editorial · Reviewed by Pedro Marini
Lead
On-device AI stopped being an academic curiosity; it's now a commercial priority. Rather than sending every request to distant servers, phones and laptops are increasingly able to run useful language and vision models locally. That matters — it changes who controls data, who pays for compute, and where the next tussles between Apple, Google and chipmakers will happen.
What actually changed
Put those pieces together and a smart assistant can parse your calendar, summarize a PDF or answer a coding question without ever leaving your phone.
Why it matters beyond privacy
Privacy grabs headlines, but the ripple effects are broader and messier.
What's interesting here is how these trade-offs get negotiated in real products — and fast.
A few concrete scenarios
These are simple examples, but they point to real product choices: when something happens locally versus when it gets escalated to cloud services.
Counterpoints and limits
Local inference is not a cure-all. Large, stateful models will still live in the cloud for scale, continual training and complex multimodal fusion. On-device models tend to lag in raw capability and require careful update strategies to avoid drift or stale facts. Pushing updates at scale — across OS versions, carriers and legacy hardware — is its own headache.
Historical parallel and editorial take
It resembles the shift from film labs to digital cameras: capabilities decentralize, empowering users and startups. But power also concentrates around the platforms that control chip design, update channels and app distribution. That concentration is worth watching; it won't necessarily play out in favor of the nimblest developer.
What to watch next
On-device AI isn't a fad. It will make many tasks faster, safer and cheaper — and it will add a layer of commercial and regulatory complexity. For investors and product people, the smarter bet is less about which model wins and more about the hardware, runtimes and distribution channels that make local intelligence practical and sustainable. Expect a messy, competitive few years — and some surprising winners.
As generative AI demands more training material, synthetic and clean-room datasets are becoming strategic assets for U.S. firms. Here’s what investors, engineers, and policy makers need to know.
Privacy-first models, local LLMs and a silicon race are changing how banks, fintechs and investors think about AI. Short latency, big consequences.
Edge models, new silicon and privacy pressure are pushing generative AI onto phones. That shift redraws winners and losers from chips to cloud, and changes how apps make money.