The first month of an AI feature costs almost nothing, which is exactly why the fourth month is a shock. Usage grows with adoption, and unlike most software costs, model spend scales linearly with success.
Teams tend to respond by asking whether there's a cheaper model. Sometimes there is. More often the cost is structural.
Four levers, in order of payoff
- Don't call the model — cache semantically similar requests and short-circuit the obvious ones
- Send less — retrieve tighter, compact prompts, drop the boilerplate nobody reads
- Route by difficulty — cheap model first, escalate only on low confidence
- Then, and only then, negotiate rates or change providers
Measure per unit of business value
Cost per request is a starting point, not the metric. Cost per resolved ticket, per processed submission, per closed quote — that's the number that tells you whether the feature works.
On one engagement, a routing change cut inference cost 62% per request with identical scores on the evaluation set. Nothing about the output changed. The architecture had simply been sending the easy 80% to the expensive model.
Budget it like capacity
Forecast token spend against your adoption curve the way you'd forecast database load. Set alerts on cost per unit, not just total spend — total spend rising while unit cost falls is a healthy business, and you want to be able to tell those apart at a glance.