To have private AI, three things have to be under your control: the data, the model, and the hardware. Most offerings that claim to be “private” only give you two.
Every AI setup has three parts: your data, the model that does the thinking, and the hardware it runs on. Private AI is not securing just one, it is whether all three sit inside your walls.
You share the model with everyone who uses it, including your rivals. The provider sets the price, the rules, and what happens with all that data you send.
Your files stay local. But every question still goes to a model that is shared with everyone else, so what the model learns from your data does not stay with you.
Nothing is shared. The model, the machines, and the files are yours. What the system learns stays inside the building. AIPod Mini sits here.
Public AI and API-driven AI are the mechanisms Nadella describes. Every prompt teaches the model something about your business; every follow-up correction teaches it more precisely. None of that comes back to you — it goes to whoever owns the model, and to every other company renting that model, including your competitors and the startups entering your market. It can even reach the foundation model itself, which could eventually starts competing with you.
That is what makes Agentic AI unlike every system before it: old IT retrieved what you gave it but it did not learn from what you asked. AI remembers in five layers. The first three can be cleared: 1) working memory within a conversation, 2) conversational memory across sessions, and 3) semantic memory, which stores meaning rather than words. The two layers that remain are behavioral memory and system-level learning, corrections folded into the model itself. Neither can be undone once written.
A product can keep your files on your own storage, run on hardware you bought, and still send every question to a model somebody else owns and operates. That is usually sold as private AI.
The vector database reads files by meaning, not keyword, finding relevant pieces even when the source is unstructured and messy.
Prompts, source data, embeddings and answers never leave your environment.
Directors already carry a duty to oversee where company data lives. That duty does not pause when an agent makes the decision instead of an employee. An agent can email a customer or approve a claim thousands of times a day, with no one reading most of it. Boards can be held responsible for that, the same way they are already held responsible for data stored in the wrong place.
Iterate builds private AI for hospitals, banks and other regulated organizations, and saw this privacy gap directly. That is why Iterate built AgentWatch: which logs every agent action, enforces policy at the gateway rather than after the fact, and gives a board a record for proper governance and oversight.
For certain use cases, shared models are not wrong to use — most companies land on a hybrid mix, about 80/20 private to public. Equinix, which operates 280+ data centers worldwide, estimates roughly 80% of workloads run more efficiently in a private environment.
Read: “The Reverse Information Paradox” — Nadella (Jul 12, 2026). Also: The Oohs, Awes, and Dangers of AI Memory — the five layers in full. Also: You Cannot Govern What You Cannot See — the case for AgentWatch. Also: Most AI Is Shared, Not Private.
A joint educational series on private AI and the AIPod Mini. NetApp — the governed data-control layer. Iterate.ai — the private intelligence layer.
