Skip to content
Underread.
Guides

How to Cut Your LLM API Bill Without Changing Models (6 Changes)

Six changes, none of which touch which model you call, so no output changes. Together they usually take about a third off.

By Vemuri AbhiramUpdated 18 November 20265 min read
Laptop screen showing analytics charts on a desk
Laptop screen showing analytics charts on a desk
On this page

The invoice jumps, and the first instinct is always the same: switch to a smaller model, take the quality hit, hope nobody notices. Do these first. None of them changes a single output.

#Find out where the money goes first

Before changing anything, pull last month's usage broken down by feature. If you can't do that, you're optimising blind, and the fix is to tag every call with a feature name and log its token counts.

Almost every small team I've looked at finds the same thing: one unglamorous background job is the majority of the bill. Usually it's an enrichment or classification task nobody thinks about because it runs at 3am and never fails.

#1. Turn on prompt caching — and order the prompt for it

If your system prompt is long and stable, caching it's close to free money. The cached portion bills at a fraction of the normal input rate. The catch is that caching keys on an exact prefix match. Putting anything variable at the top of the prompt invalidates the cache on every single call. The usual culprits:

  • a timestamp or today's date
  • a user id or session id
  • a randomised greeting

I order mine stable-first: system instructions, then examples, then the variable user content last.

#2. Cap max_tokens honestly

An uncapped completion is a blank cheque. Cap it. If your answers run 2 paragraphs, cap at what 2 paragraphs cost. You're not restricting the model; you're refusing to reserve budget you never use.

#3. Stop resending history nobody reads

Agent frameworks default to sending the entire message list every turn. By turn 15 you're paying to re-transmit turns one through fourteen, every time, forever. Keep the system prompt, the original request and the last few exchanges. Summarise the middle into one message.

#4. Batch anything that's not interactive

The Batch API trades latency for a significant discount. Anything on a schedule — enrichment, classification, summarising yesterday's tickets — has no reason to run synchronously and compete with your live traffic.

#5. Deduplicate before you call

Run the obvious check: are you asking the same question twice? A content hash in front of the API catches more repeats than most teams expect, especially inside retry paths where the same prompt gets resent after a timeout that actually succeeded.

#6. Move the easy calls off the model entirely

Not everything needs a model:

Task What teams reach for What actually works
"Is this a valid email address?" An LLM call A regular expression
"Which of these five categories?" An LLM call A classifier, or keywords
"Extract the invoice total" An LLM call A template, when the format is fixed
"Summarise this complaint" An LLM call An LLM call

The last row matters. The point isn't that models are overkill — it's that mixing deterministic work into model calls makes both slower and more expensive than running them separately.

#What each change is worth

Change Typical saving Effort Changes output?
Reorder prompt for caching Large 10 min No
Cap max_tokens Medium 10 min No
Stop resending history Medium to large 1 hour No
Batch scheduled work Large on that job Half a day No
Deduplicate Varies wildly 2 hours No
Move easy calls off the LLM Medium Varies Improves it
Downgrade the model Large 10 min Yes

Every row above the line is free in quality terms. That's the whole argument for doing them first.

#Where this usually lands

Prompt reordering plus caching, a real max_tokens cap, and scheduled work moved to batch is a reduction of roughly a third on a typical workload. Outputs are identical. Do those 3 before you consider downgrading anything.

#What to expect in the first month

Give it 30 days before you judge any of this. Billing lags. Caching takes a few days to settle, and the first week after you reorder a prompt often looks worse than it is. Your cache is cold (nothing matches yet, so everything bills at full rate). What you should see by week 3 is input cost falling faster than output cost, which tells you the caching is landing. If your bill hasn't moved at all by then, something variable is still sitting at the top of your prompt. That's where I'd look first.

Related reading: fixing rate limit errors — the same changes usually fix both — and context length errors.

Frequently asked questions

Is a cheaper model really that much worse?

For extraction and classification, often not. For nuanced writing and multi-step reasoning the gap is still obvious. Test on your own data before deciding.

How do I see where the money actually goes?

Tag every call with a feature name and log token counts per tag. Most teams find one background job is the majority of the bill.

Does prompt caching change my outputs?

No. It changes how the input is billed, not what the model produces.

Why did caching not reduce my bill?

Almost always because something variable sits at the top of the prompt, so the cached prefix never matches. Move dates, ids and random values to the end.

Written by

Vemuri Abhiram

Founder & Editor

I write every post here. The tools I review are ones I pay for and run on real client work, not demo accounts, and every number you see came off my own invoice or dashboard.

More from Underread, once a week.

The guides worth reading and the tests worth trusting. One email, every Friday.

No spam. Unsubscribe in one click.