Seven ways to cut LLM API costs, in order of effect
When the bill surprises you, work down this list rather than starting at the bottom.
AI Tools Hub Editorial · published · updated
In order
| Measure | Effect | Effort |
|---|---|---|
| 1. Use prompt caching | Large | Low |
| 2. Shorten the input | Large | Medium |
| 3. Different models per task | Large | Medium |
| 4. Cache repeated queries in your app | Medium–large | Medium |
| 5. Move work to batch APIs | Medium | Low |
| 6. Cap output length | Medium | Low |
| 7. Switch everything to a cheaper model | Medium | High — needs quality testing |
Caching needs the prefix stable
Prompt caching only works when the cached portion is at the start of the request and matches exactly. Implementations that put user input early defeat it. Reorder before concluding caching does not help.
Shorter inputs, same quality
If RAG passes twenty chunks, try five. Quality usually holds, because beyond a point extra chunks add noise rather than context. Conversation history is the other common waste — send the recent turns plus a summary instead of everything.
Build an evaluation set before step 7
Twenty to thirty representative inputs with expected outputs, run against both models. It takes an afternoon and turns every future model decision into a measurement rather than a guess.
FAQ
Where do I start?
Measure first. Per-task token counts almost always show one operation consuming most of the budget, and fixing that one is usually enough.