How to Cut Your LLM API Bill Without Changing Models (6 Changes)
Six changes, none of which touch which model you call, so no output changes. Together they usually take about a third off.
On this page
- Find out where the money goes first
- 1. Turn on prompt caching — and order the prompt for it
- 2. Cap max_tokens honestly
- 3. Stop resending history nobody reads
- 4. Batch anything that's not interactive
- 5. Deduplicate before you call
- 6. Move the easy calls off the model entirely
- What each change is worth
- Where this usually lands
- What to expect in the first month
The invoice jumps, and the first instinct is always the same: switch to a smaller model, take the quality hit, hope nobody notices. Do these first. None of them changes a single output.
#Find out where the money goes first
Before changing anything, pull last month's usage broken down by feature. If you can't do that, you're optimising blind, and the fix is to tag every call with a feature name and log its token counts.
Almost every small team I've looked at finds the same thing: one unglamorous background job is the majority of the bill. Usually it's an enrichment or classification task nobody thinks about because it runs at 3am and never fails.
#1. Turn on prompt caching — and order the prompt for it
If your system prompt is long and stable, caching it's close to free money. The cached portion bills at a fraction of the normal input rate. The catch is that caching keys on an exact prefix match. Putting anything variable at the top of the prompt invalidates the cache on every single call. The usual culprits:
- a timestamp or today's date
- a user id or session id
- a randomised greeting
I order mine stable-first: system instructions, then examples, then the variable user content last.
#2. Cap max_tokens honestly
An uncapped completion is a blank cheque. Cap it. If your answers run 2 paragraphs, cap at what 2 paragraphs cost. You're not restricting the model; you're refusing to reserve budget you never use.
#3. Stop resending history nobody reads
Agent frameworks default to sending the entire message list every turn. By turn 15 you're paying to re-transmit turns one through fourteen, every time, forever. Keep the system prompt, the original request and the last few exchanges. Summarise the middle into one message.
#4. Batch anything that's not interactive
The Batch API trades latency for a significant discount. Anything on a schedule — enrichment, classification, summarising yesterday's tickets — has no reason to run synchronously and compete with your live traffic.
#5. Deduplicate before you call
Run the obvious check: are you asking the same question twice? A content hash in front of the API catches more repeats than most teams expect, especially inside retry paths where the same prompt gets resent after a timeout that actually succeeded.
#6. Move the easy calls off the model entirely
Not everything needs a model:
| Task | What teams reach for | What actually works |
|---|---|---|
| "Is this a valid email address?" | An LLM call | A regular expression |
| "Which of these five categories?" | An LLM call | A classifier, or keywords |
| "Extract the invoice total" | An LLM call | A template, when the format is fixed |
| "Summarise this complaint" | An LLM call | An LLM call |
The last row matters. The point isn't that models are overkill — it's that mixing deterministic work into model calls makes both slower and more expensive than running them separately.
#What each change is worth
| Change | Typical saving | Effort | Changes output? |
|---|---|---|---|
| Reorder prompt for caching | Large | 10 min | No |
Cap max_tokens |
Medium | 10 min | No |
| Stop resending history | Medium to large | 1 hour | No |
| Batch scheduled work | Large on that job | Half a day | No |
| Deduplicate | Varies wildly | 2 hours | No |
| Move easy calls off the LLM | Medium | Varies | Improves it |
| Downgrade the model | Large | 10 min | Yes |
Every row above the line is free in quality terms. That's the whole argument for doing them first.
#Where this usually lands
Prompt reordering plus caching, a real max_tokens cap, and scheduled work moved to batch is a reduction of roughly a third on a typical workload. Outputs are identical. Do those 3 before you consider downgrading anything.
#What to expect in the first month
Give it 30 days before you judge any of this. Billing lags. Caching takes a few days to settle, and the first week after you reorder a prompt often looks worse than it is. Your cache is cold (nothing matches yet, so everything bills at full rate). What you should see by week 3 is input cost falling faster than output cost, which tells you the caching is landing. If your bill hasn't moved at all by then, something variable is still sitting at the top of your prompt. That's where I'd look first.
Related reading: fixing rate limit errors — the same changes usually fix both — and context length errors.
Frequently asked questions
Is a cheaper model really that much worse?
For extraction and classification, often not. For nuanced writing and multi-step reasoning the gap is still obvious. Test on your own data before deciding.
How do I see where the money actually goes?
Tag every call with a feature name and log token counts per tag. Most teams find one background job is the majority of the bill.
Does prompt caching change my outputs?
No. It changes how the input is billed, not what the model produces.
Why did caching not reduce my bill?
Almost always because something variable sits at the top of the prompt, so the cached prefix never matches. Move dates, ids and random values to the end.
Written by
Vemuri Abhiram
Founder & Editor
I write every post here. The tools I review are ones I pay for and run on real client work, not demo accounts, and every number you see came off my own invoice or dashboard.
You might also like
How to Fix “Rate limit reached for requests” on the OpenAI API (6 Fixes)
The OpenAI rate limit error, how to tell which of the three limits you hit, and six fixes in the order that resolves it fastest.
Vemuri Abhiram6 min read
How to Fix “This model's maximum context length is…” (4 Fixes)
The context length error explained, plus four fixes ranked by how much answer quality each one costs — and why a bigger window should be last.
Vemuri Abhiram5 min read
OpenAI Cuts API Prices Again: What Changes for Small Builds
What another round of API price cuts actually changes for small teams, and the two changes that matter more than the headline rate.
Vemuri Abhiram2 min read
More from Underread, once a week.
The guides worth reading and the tests worth trusting. One email, every Friday.
No spam. Unsubscribe in one click.