The free sandbox closed
On September 3, 2026, the General Services Administration started charging agencies for USAi, the shared generative AI platform it launched as a free service a year earlier. The new model is a platform fee sized to each agency's expected workload plus model usage passed through at cost, according to Nextgov's report. About 25 agencies were on the platform in June, with 16 more onboarding by year end.
That is a small administrative change with a large meaning. The experimentation phase of federal AI had a sponsor. The production phase has a meter. Every organization that has run a pilot on someone else's budget, a vendor credit, a free tier, or an innovation fund, reaches the same day, and the question that arrives with it is not whether the AI works. It is what the AI costs per month, per decision, and per year, and who signs for it.
Most organizations cannot answer that yet. Brookings counted more than 3,600 federal AI use cases across 41 agencies in the 2025 inventories, and about three in five of them are still in pilot or pre-deployment. The dollars have already moved: Brookings' separate contracting analysis puts obligated federal AI funds at $7.2 billion in 2026 across 1,743 awards, up from $675 million and 489 awards in 2024. And when GAO reviewed 13 AI acquisitions this spring in GAO-26-107859, agency officials told the auditors that one of their persistent obstacles was simply understanding AI-related cost structures.
The commercial side is not further ahead. It is just less audited.
Cheaper tokens, bigger bills
The counterintuitive fact of 2026 is that unit prices collapsed and total spend rose. Public cloud API pricing for language models fell nearly 80 percent year over year, and enterprise AI budgets grew anyway, which CIO's Steve Oppenheim summarizes as a volume problem, not a unit cost problem. The FinOps Foundation's data set of 1,192 organizations has AI workloads at 18 percent of cloud spend at AI-forward enterprises, up from 4 percent in 2023.
The reason is structural. A pilot runs when someone demonstrates it. A production system runs every time the workflow runs, around the clock, with retries, with retrieval, and increasingly with agents that make several model calls to complete one task. Gartner expects inference to account for at least 70 percent of a model's lifetime cost, and predicts that at least half of generative AI initiatives will overrun their budgets by 2028. The stated causes are worth reading twice: poor architectural choices and a lack of operational know-how. Not model pricing. Design and operations.
So the line item everyone negotiates, the per-token or per-seat price, is the one that keeps getting cheaper. The lines nobody put in the pilot budget are the ones that grow.
Where the money actually goes
Across the AI systems we build and operate, production cost settles into five lines. The model is one of them, and over a multi-year horizon it is rarely the largest.
Inference at real volume. Not the demo's hundred documents but the year's hundred thousand, multiplied by retries, by the retrieval calls that feed the model its context, and by every agent step that turns one request into six. This line is measurable from day one if the system counts calls per unit of work. Most pilots do not count.
Evaluation. A frozen test set built from real cases, a harness that runs it, and the discipline to run it again every time a model, a prompt, or a retrieval index changes. Model providers retire and replace versions on their own schedule. Without an evaluation set the upgrade is a gamble; with one it is a Tuesday. This line is small in dollars and enormous in consequence, and it is the first thing cut when the pilot budget is written.
Data plumbing. Getting the documents, records, and permissions into a shape the system can use, and keeping them there as the source systems change underneath. In our experience this is the largest engineering line in the first year of almost every project, and it is the one that looks like it belongs to somebody else's budget.
Monitoring and traceability. Logs that keep the input, the model version, the retrieved sources, and the output together for every decision, retained long enough that a person can reconstruct a case months later. Storage is cheap. Designing the lineage in is cheap. Retrofitting it onto a running system is one of the most expensive things you can do to software.
People who own it. Someone who watches the dashboards, reads the flagged outputs, owns the evaluation set, and decides when the model changes. KPMG's Q2 2026 pulse, cited in Flexera's 2026 analysis, found that only 24 percent of organizations have executive-level accountability for AI, and that the ones with clear ownership report three times the ROI. The same analysis has 59 percent of organizations reporting that wasted AI spend rose year over year, and only 31 percent with accurate visibility into what their AI software costs. Those three numbers describe one condition: nobody owns the meter.
Budget it as a system you operate
The fix is not a bigger number. It is a different shape of number. Three moves change how the budget behaves once the system is live.
Price per decision, not per seat. The unit that matters is the unit of work: a document processed, a case resolved, an application screened. Put every cost that scales with that unit into one figure, including the model calls, the retrieval, the retries, and the human review minutes it triggers. A cost per decision is something a COO can compare against the manual process it replaces. A monthly platform fee is not.
Let the architecture cap the bill. The single largest lever on inference cost is deciding what the model is allowed to do. Where a rule exists, the rule runs in code that can be tested for free. Where a document has to be read, the model reads it, and the extracted figures land in deterministic logic that does the arithmetic. Small tasks route to small models. Repeated questions hit a cache. On XCreos, the AI underwriting platform we designed and built for commercial real estate, this split was a trust decision before it was a cost decision: the model reads the rent roll and the operating statement, and every figure it extracts traces to the line it came from, while the underwriting math runs in code. The side effect is that the expensive part of the pipeline runs once per document, and the rest runs for the price of a database query.
Fund year two before you fund year one. A pilot budget covers a build. A production budget covers a build and then an operation, and the operation is the part that is still there in eighteen months when the vendor has replaced the model twice and the source system has changed its export format. If the line for evaluation, monitoring, and a named owner is zero, the system has an expiration date that nobody wrote down.
This is also why we structure engagements the way we do. The team that designs and builds a platform stays to operate and improve it. That is not a support contract. It is the year-two line, priced honestly and owned by the people who know where the bodies are.
A worksheet you can defend
Before the budget conversation, answer these in writing. Every one of them can be checked inside a live system rather than in a proposal.
- What is the unit of work, and what does one unit cost today, all in: model calls, retrieval, retries, and review time?
- How many model calls does one unit make at the 50th and the 95th percentile? If nobody knows, the system is not counting, and the bill will be a surprise.
- Where is the evaluation set, who owns it, and what did it cost to run the last time the model changed?
- What is logged per decision, for how long, and what does that storage cost at next year's volume?
- Who reads the monitoring, what fraction of their week is that, and what happens on the day they leave?
- What did the manual process cost per unit, measured the same way? That is the number the AI cost has to beat, and the only one that makes the case.
A vendor or a team that built for production answers these from the product and the dashboards. One that built a pilot answers them from the roadmap. The difference is audible in about ten minutes, and it is the difference between the organizations that cross into production and the three in five that stay in the sandbox after the sandbox starts sending invoices.
Where this starts
The cheapest moment to get the budget right is before the architecture is chosen, because the architecture is what decides whether the bill scales with your business or with your users' curiosity. Our Assessment produces that decision in two to four weeks: what the unit of work is, what it costs by hand today, what the system should decide and what it should only prepare, and a cost model for years one and two that a finance team can sign.
Sometimes the finding is that the AI pays for itself many times over. Sometimes it is that a smaller model, or no model at all, does the job for a tenth of the price. Either way you get a number you can defend, which is the only kind worth putting in front of the people who approve it.
The token got cheap. The system did not. Budget for the system.