Where AI Spend Leaks
These are the main places we look for savings. Each one has a quality risk, and we say what it is. Vendor pricing changes often. Check current pricing for your provider and model.
| Lever | What it is | Quality risk |
|---|---|---|
| Model routing | Send simple requests to smaller models | Medium |
| Prompt caching | Reuse repeated prompt prefixes | Low |
| Batching | Use batch APIs for non-urgent work | Low |
| Context trimming | Send less text per request | Medium |
| Output limits | Cap response length | Low to medium |
| Retries and loops | Remove runaway repeat calls | Low |
| Duplicate calls | Stop asking the same thing twice | Low |
| Result caching | Store and reuse answers | Medium |
| Model swaps | Move to a cheaper model | Medium to high |
Model routing
Not every request needs your largest model. Classification, formatting and short lookups often work on a smaller one. The risk is that the router sends a hard request to a weak model. We test this on real examples first.
Prompt caching
Vendors state that repeated prompt prefixes can be billed at a discount. Whether this applies depends on provider and model. Quality risk is low because the model sees the same text. The saving depends on how much of your prompt repeats.
Batching
Vendors state that batch endpoints for non-urgent work are priced at roughly half of standard rates. Results come back later, not instantly. It suits reports, back-office jobs and data processing. It does not suit live chat.
Context trimming
Prompts often carry old history or unused documents. Cutting them lowers cost. The risk is removing something the model needed. We test before and after.
Output limits
Long answers cost more. Setting sensible limits helps. The risk is cut-off answers, so we check where limits bite.
Retries and loops
A failing call that retries many times, or an agent that keeps looping, can burn money fast. Fixing these has little quality risk and often large effect.
Duplicate calls
The same input is sometimes sent several times by different parts of a system. Removing the duplicates saves cost with no change to output.
Result caching
If many users ask the same question, the stored answer can be reused. The risk is stale answers. We set expiry rules.
Evaluation before model swaps
Moving to a cheaper model can save money, but quality can drop in ways that are hard to see. We build a small evaluation set from your real cases and compare results before any swap. If quality falls, we do not recommend it.
No promises
We do not claim a set percentage. What is possible depends on your usage.