The harder your AI works when using Public AI or shared models, the larger the bill grows.
Public AI charges you like a taxi meter that never stops running. Every question and every extra step the AI takes to think adds a few more cents to the fare. The busier your AI gets, the faster that meter spins: the more work it does, the bigger the bill. Own the car instead, and the meter disappears for good.
Agentic AI can turn a modest AI expense into a serious operating cost surprisingly quickly. A task that once cost $0.04 per interaction can reach $1.20 when expanded into an agentic workflow—a 30× increase. At larger scale, the numbers become stark: enterprise AI systems can consume billions or even tens of trillions of tokens per month, with one recent internal usage tally reaching 73.7 trillion tokens in 30 days, equivalent to roughly $221 million at list-price economics.
The issue is not simply token price; it is token burn rate. Agentic workflows can consume hundreds of thousands of tokens in a single session, while individual users can generate hundreds of millions of tokens annually. Research also finds that the same agentic task can vary by as much as 30× in token consumption. Without controls around model selection, context, retries, and token budgets, the AI equivalent of a taxi meter can keep running long after anyone is watching the fare.
Per-token prices have fallen up to 98% a year (÷200 since 2024) — yet total AI bills keep tripling. The price cut never reaches your invoice, because three forces multiply against it:
Illustrative — framework adapted from Agora Software; price data: Epoch AI. Usage multipliers vary by workload.
Cheaper tokens just unlock more use: most of the spend is agent orchestration. The cost sits less in the prompting and more in the agent work and inference process — roughly a 1:11 ratio.
A chatbot answers once. An agent plans, fetches, checks, retries — every hidden step is another billed event.
Steps 2 to 4 repeat, often dozens of times. You see step 1 and step 5 and pay for everything in between — where nearly all the tokens go.
One request becomes dozens of model calls — 5–30× the tokens of a single chat, sometimes far more.
The model keeps no state between calls, so context is re-sent or re-fetched each step. Nothing is forgotten — you pay to reload it every turn.
User habits and a single pricing rule compound costs: simple jobs get sent to premium models that cost more than the task needs, and the AI’s answers (output tokens) bill several times higher than your prompt (input).
A session grows from ~2K to ~25K tokens per call (Mem0). You pay for the prompt, answer, and everything in between.
On hardware you own, token cost stops mattering: the agent loops as often as needed, and the invoice is the same.
Estimated private AI TCO is ~$118K in year one (then ~$53K/yr) vs ~$564K in ongoing public AI token costs.
Full paper, 12 pages — iterate.ai/partners/netapp/private-ai-education-and-certification-program.
The economics:
“The Token Tax” and
“The Death of the Token Tax” Iterate.ai.
Also: FinOps Foundation, 2026 State of FinOps.
A joint educational series on private AI and the AIPod Mini — from NetApp and Iterate.ai.
