How to Fix “This model's maximum context length is…” (4 Fixes)
Four ways to get under the limit, ranked by how little each one costs you in answer quality. The cheapest fix is usually the one nobody checks.
On this page
The call fails with some version of this:
This model's maximum context length is 128000 tokens.
However, your messages resulted in 131204 tokens.
The model isn't confused and nothing is broken. You sent more than it can hold in one go.
#Measure before you cut
Guessing token counts from character counts is how you end up 3,000 over and confused. A token is roughly 4 characters of English, but code, names and non-English text break that rule badly.
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
print(len(enc.encode(text)))
2 things people forget when they do the sum:
- The budget is shared. Your input plus
max_tokensfor the reply must both fit inside the window. - Tool definitions count. A dozen functions with detailed descriptions can be several thousand tokens before the conversation starts.
#1. Stop resending the whole conversation
This is the most common cause by far, and the cheapest to fix. An agent loop that appends every tool call and every tool result to the message list will exhaust any window given enough turns. The fix is to keep what matters and summarise the rest: hold the system prompt, the original user request, and the last few exchanges verbatim; compress everything between them into one short message.
Quality cost: low. Most of those middle turns were mechanical.
#2. Retrieve fewer chunks, not shorter questions
If you're doing RAG (retrieval-augmented generation — pulling relevant documents in before asking) and hitting the limit, you're retrieving too much, not asking too much. Dropping from 20 chunks to 8 changes answer quality far less than people fear. It's also the difference between a call that works and one that errors. If 8 genuinely isn't enough, your chunks are too small or your retrieval is ranking badly, and that's the real bug.
Quality cost: low to medium, and often negative — fewer, better chunks beat more, noisier ones.
#3. Summarise the middle
For a single long document, a two-pass approach holds up well. Summarise each section, reason over the summaries, then pull the full text of only the two or 3 sections that turned out to matter.
Quality cost: medium. You're trading detail for reach.
#4. Move to a bigger window — last
| Fix | Quality cost | Money cost | Effort |
|---|---|---|---|
| Trim history | Low | Saves money | 1 hour |
| Fewer RAG chunks | Low to medium | Saves money | 1 hour |
| Summarise the middle | Medium | Slightly more | Half a day |
| Bigger context model | None | More, on every call | 10 minutes |
A bigger window is the fastest to implement and the most expensive to live with, because the cost recurs on every single call from then on. Try the other three first.
#Match your symptom to the fix
| What you're seeing | Most likely cause | Go to |
|---|---|---|
| Fails after ten or so agent turns | History resent every turn | Fix 1 |
| Fails on some documents, not others | Retrieval pulling too many chunks | Fix 2 |
| Fails on one enormous PDF | Single document larger than the window | Fix 3 |
| Fails on the very first call | Tool definitions plus system prompt too large | Measure first |
| No error, but answers got vague | Context too long to attend to properly | Fix 2 or 3 |
That last row is the one that catches people. Fixing the error by enlarging the window can quietly replace a loud failure with a silent one.
#When none of them are enough
If a single indivisible document genuinely exceeds the largest window you can afford, the shape of the problem has changed: you need retrieval over that document rather than one pass through it. Chunk it, index it, and ask targeted questions. That's more work than any fix above, which is why it comes last.
#What you should check before you ship the fix
Once you're under the limit, run 20 of your real inputs through and read the answers rather than the status codes. Context errors usually hide a second problem — your retrieval is pulling 15 chunks where 6 would do, or your agent has been resending 40 turns of history since the day you built it. The error was the only thing telling you. If your answers were already vague before the error appeared, trimming won't help. Look at how your chunks are being ranked instead.
Related reading: how to fix OpenAI rate limit errors, which the same workloads tend to trigger, and cutting your LLM bill.
Frequently asked questions
Do images count toward the context limit?
Yes. Vision inputs are tokenised and can be expensive, often several hundred tokens per image depending on resolution.
Does the system prompt count?
Every message counts, including the system prompt and every tool definition you pass with the request.
Is a bigger context window always better?
No. Retrieval accuracy drops in the middle of very long contexts, so a well-trimmed short context often outperforms a stuffed long one.
How do I count tokens without calling the API?
Use tiktoken locally for OpenAI models. It runs offline and matches the server's count closely enough to budget against.
Written by
Vemuri Abhiram
Founder & Editor
I write every post here. The tools I review are ones I pay for and run on real client work, not demo accounts, and every number you see came off my own invoice or dashboard.
You might also like
How to Fix “Rate limit reached for requests” on the OpenAI API (6 Fixes)
The OpenAI rate limit error, how to tell which of the three limits you hit, and six fixes in the order that resolves it fastest.
Vemuri Abhiram6 min read
How to Cut Your LLM API Bill Without Changing Models (6 Changes)
Six ways to cut an LLM API bill that do not involve downgrading the model, ordered by how much they typically save.
Vemuri Abhiram5 min read
OpenAI Cuts API Prices Again: What Changes for Small Builds
What another round of API price cuts actually changes for small teams, and the two changes that matter more than the headline rate.
Vemuri Abhiram2 min read
More from Underread, once a week.
The guides worth reading and the tests worth trusting. One email, every Friday.
No spam. Unsubscribe in one click.