I sat in a retro for a FinOps dashboard my team had built three months earlier. Two hundred charts, every service down to the training job. Attendance: two people, both of us from the FinOps team. The engineer whose service had triggered the anomaly the whole dashboard existed to catch had never logged in.
That dashboard reported total spend, savings identified, tagging coverage, and cost per instance, which is what the cloud bill and finance’s spreadsheet can produce. None of them changed a decision, because none was in a unit the product team decides in. FinOps KPIs for AI products have one test: did someone change a model, a prompt, a price, or a forecast because of this number. Twelve KPIs went through that test on production AI products. Six survived.
The direct version: the FinOps KPIs that hold up on AI products are cost per request, cost per active user, cost per subscriber, cache hit rate, unit cost trend, and forecast variance. Total spend, savings identified, tagging coverage, commitment coverage, cost per instance, and anomaly count were dropped from the AI dashboard because, on a Fortune-500 financial data platform with eight figures of annual AI and cloud spend, nobody changed a decision on them.
FinOps KPIs for AI products: what survived
| KPI | What it measures | Decision it changed | Verdict |
|---|---|---|---|
| Cost per request | Model, retrieval, and eval spend per call, by feature | Model choice, prompt length | Kept |
| Cost per active user | Cost per request times requests per user | Budget, pacing | Kept |
| Cost per subscriber | Cost per active user times activation | Pricing | Kept |
| Cache hit rate | Share of input served from cache | Architecture, planning price | Kept |
| Unit cost trend | Cost per request, month over month, by feature | Re-measure triggers | Kept |
| Forecast variance | Forecast versus actual, by driver | Trust in the number | Kept |
| Total spend | The bill | None on its own | Dropped |
| Savings identified | Recommendations nobody actioned | None | Dropped |
| Tagging coverage | Share of resources tagged | None for inference | Dropped |
| Commitment coverage | Share of spend on commits | Infrastructure only | Moved |
| Cost per instance | Compute cost per node | None for inference | Dropped |
| Anomaly count | Alerts fired | None; alerts on total spend | Dropped |
The six that survived
Cost per request (cost per query, cost per call)
Model, retrieval, and eval spend attributed to a feature, divided by the requests it served, from the vendor’s usage export rather than inferred from compute. Features on the same model differed ten to one, which is the finding that made every other KPI on this list possible.
Caveat: an all-up cost per request across a product hides the feature losing money. Report it per feature or not at all.
Cost per active user (cost per MAU)
Cost per request times requests per active user per month. The second input is the one launch estimates skip and the one that moved most: it doubled within two quarters of shipping. This is the KPI budgets and pacing run on, because it moves with adoption in a way total spend cannot explain.
Caveat: “active” needs one definition finance and product share, or two teams report two numbers from the same data.
Cost per subscriber (cost per seat)
Cost per active user times the share of subscribers who are active. The one KPI in the same unit as revenue, so the one that sits next to the seat price in a pricing review. A costly feature at 20 percent activation can be cheaper per seat than a modest feature everyone uses.
Caveat: activation rises after launch, so cost per subscriber drifts upward with no change in cost per request.
Cache hit rate
The share of input tokens served from cache. Under 30 percent, the cached price is a rounding error. Above 60 percent, pricing on uncached rates overstates cost per request by a third or more. It survived because it tells the product team which price to plan on.
Caveat: the hit rate in a test harness with one user is not the hit rate with a thousand.
Unit cost trend (cost per request, month over month)
Cost per request for each feature, tracked over time, with a flag when it moves. On the same deployed model with the same instance count, cost per request moved by more than half within a quarter after a prompt template changed. Nothing infrastructural happened; the trend line was the only place it showed.
Caveat: a prompt edit is a pricing event. The trend is useless unless prompt changes are logged next to it.
Forecast variance (forecast accuracy)
Forecast against actual, by driver. Driver-based forecasting landed within 5 percent of actual on a nine-figure multi-cloud estate. An earlier forecast held within 2 percent for three quarters and missed by almost thirty in the fourth, because a feature ran ten times the inference per session and the model still counted cost per instance.
Caveat: variance on the total hides offsetting misses. Report it per driver.
The six that were dropped
Total spend (the cloud bill)
The number every dashboard starts with and the one that changed nothing on its own. Total spend rises with adoption, which is what the product is for. It stayed on finance’s view as context and came off the product view.
Caveat: it still matters for pacing against a budget, and that is a cost per active user problem wearing a total.
Savings identified (optimization recommendations)
The sum of what a tool says could be saved. On compute, a rightsizing backlog. On inference, a list nobody owns, because the levers (caching, routing, prompt length) belong to product engineering and the tool cannot see them.
Caveat: savings realized is a different metric and a real one. Savings identified is a pipeline nobody drains.
Tagging coverage (allocation coverage)
Share of resources carrying a cost tag. Reaches 95 percent on compute and 0 percent on a shared model endpoint, because the endpoint is one resource serving five features. Replaced by request labeling, which is a coverage metric at the call level.
Caveat: keep it for the cloud side around the model: the retrieval index, the vector database, the eval pipeline.
Commitment coverage (reserved capacity, committed use)
Share of spend covered by a commitment. Moved rather than dropped: it governs the provisioned throughput pool on the infrastructure view, and it has no meaning per feature. What the feature sees is its share of the pool, allocated by throughput.
Caveat: at low utilization a request on a provisioned pool ran two to three times the pay-per-token equivalent, which coverage percent cannot show.
Cost per instance (cost per node, cost per GPU hour)
The easiest number to pull from a billing export and the least connected to anything the product decides. Inference cost moves on token split, cache hit rate, and retries while instance count sits still.
Caveat: still the right KPI for a training cluster. It came off the inference view, not the platform view.
Anomaly count (alerts fired)
Number of cost alerts in a period. On total spend, it fires on every legitimate scale-up or stays silent while one feature’s unit cost climbs. Dropped as a KPI, kept as a control, re-pointed at cost per request per feature.
Caveat: anomaly detection bought before allocation is fixed has nothing to define “normal” against.
Which KPI for which decision
Model choice and prompt length: cost per request. Budget and pacing: cost per active user. Pricing: cost per subscriber. Architecture and planning price: cache hit rate. When to re-measure: unit cost trend. Whether anyone trusts the rest: forecast variance. A KPI that maps to no decision on this list is a chart nobody will open.
Pre-build versus in production
Before launch, cost per request and cost per active user come from a model with assumed token split, cache hit rate, and requests per user. After launch they come from the usage export joined to product analytics. The pre-build number is for choosing a model and an architecture; the production number is the only one fit for a forecast. Cache hit rate is the KPI most likely to be wrong pre-build; plan on the uncached price until a month of production data says otherwise.
FinOps best practices that carried over
Three practices from cloud FinOps held on AI products without change. Showback before chargeback: a team that has watched its own number for two quarters accepts it. Put the signal where people already work, not in a separate portal. Assign the cost to whoever holds the lever. The FinOps Foundation’s framework covers all three, and the sibling piece on what changes when the workload is inference covers the assumptions that did not carry.
The mistake: reporting the savings backlog as a KPI
The mistake is putting savings identified on the executive view because it is a large number that looks like progress. It is a list of things no one has done. The symptom: a savings figure that grows every quarter while cost per request per feature is not on the dashboard at all. The first feature-level breakdown on that financial data platform found one of the top three AI products running several times the portfolio average; the savings backlog had no line for it because no tool could see it.
The rule
A FinOps KPI earns its place by naming the decision it changed last quarter. If it cannot, it is a chart. Six KPIs on the AI view, each with a decision beside it, in a tool finance already opens. That is what FP&A ran unassisted after the provider console came off the reporting path, and why the second dashboard’s retro had more than two people in it.
FAQ
What are FinOps KPIs?
Metrics a FinOps practice uses to make cloud and AI spend decidable: unit costs such as cost per request or per subscriber, efficiency rates such as cache hit rate, and accuracy measures such as forecast variance. The FinOps Foundation groups them under its inform, optimize, and operate phases. For AI products the unit costs matter most, because inference spend follows usage rather than infrastructure.
What are cloud cost optimization metrics for AI workloads?
Cache hit rate, cost per request by feature, retry rate, tokens per session, and the model mix per feature. Each maps to a lever the product team holds: caching, prompt length, model routing, context trimming. Compute metrics such as cost per instance and reserved coverage still apply to training clusters and provisioned pools, not to per-feature inference cost.
What are the main cloud cost drivers for AI products?
Input tokens, output tokens, cached reads, retries, tool calls, retrieval, eval overhead, and the share of any provisioned throughput pool. On production products, the ride-alongs beyond tokens added 30 to 80 percent on top of the tokens-times-price estimate. Training runs and inference traffic, not the surrounding data pipeline, were the dominant drivers on the platforms I have measured.
What to do next
Take the dashboard your team has now, write the decision each chart changed last quarter next to it, and delete the ones with a blank. When what remains needs to become a reporting path finance runs alone, that is FinOps consulting for AI products. The controls around these KPIs are in cloud cost governance for AI spend; the rest of the FinOps for AI practice is on the home page.
I put a number on what an AI product costs per request, per active user, and per subscriber, early enough to change the model, the architecture, or the price. If you’re shipping something with a model behind it and nobody can tell you what it costs, email me.
Have a number nobody can explain?
Send a note and I will get back to you.