Skip to content
Underread.
Guides

How to Fix “This model's maximum context length is…” (4 Fixes)

Four ways to get under the limit, ranked by how little each one costs you in answer quality. The cheapest fix is usually the one nobody checks.

By Vemuri AbhiramUpdated 18 November 20265 min read
Open laptop on a desk beside a notebook and coffee
Open laptop on a desk beside a notebook and coffee
On this page

The call fails with some version of this:

This model's maximum context length is 128000 tokens.
However, your messages resulted in 131204 tokens.

The model isn't confused and nothing is broken. You sent more than it can hold in one go.

#Measure before you cut

Guessing token counts from character counts is how you end up 3,000 over and confused. A token is roughly 4 characters of English, but code, names and non-English text break that rule badly.

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
print(len(enc.encode(text)))

2 things people forget when they do the sum:

  • The budget is shared. Your input plus max_tokens for the reply must both fit inside the window.
  • Tool definitions count. A dozen functions with detailed descriptions can be several thousand tokens before the conversation starts.

#1. Stop resending the whole conversation

This is the most common cause by far, and the cheapest to fix. An agent loop that appends every tool call and every tool result to the message list will exhaust any window given enough turns. The fix is to keep what matters and summarise the rest: hold the system prompt, the original user request, and the last few exchanges verbatim; compress everything between them into one short message.

Quality cost: low. Most of those middle turns were mechanical.

#2. Retrieve fewer chunks, not shorter questions

If you're doing RAG (retrieval-augmented generation — pulling relevant documents in before asking) and hitting the limit, you're retrieving too much, not asking too much. Dropping from 20 chunks to 8 changes answer quality far less than people fear. It's also the difference between a call that works and one that errors. If 8 genuinely isn't enough, your chunks are too small or your retrieval is ranking badly, and that's the real bug.

Quality cost: low to medium, and often negative — fewer, better chunks beat more, noisier ones.

#3. Summarise the middle

For a single long document, a two-pass approach holds up well. Summarise each section, reason over the summaries, then pull the full text of only the two or 3 sections that turned out to matter.

Quality cost: medium. You're trading detail for reach.

#4. Move to a bigger window — last

Fix Quality cost Money cost Effort
Trim history Low Saves money 1 hour
Fewer RAG chunks Low to medium Saves money 1 hour
Summarise the middle Medium Slightly more Half a day
Bigger context model None More, on every call 10 minutes

A bigger window is the fastest to implement and the most expensive to live with, because the cost recurs on every single call from then on. Try the other three first.

#Match your symptom to the fix

What you're seeing Most likely cause Go to
Fails after ten or so agent turns History resent every turn Fix 1
Fails on some documents, not others Retrieval pulling too many chunks Fix 2
Fails on one enormous PDF Single document larger than the window Fix 3
Fails on the very first call Tool definitions plus system prompt too large Measure first
No error, but answers got vague Context too long to attend to properly Fix 2 or 3

That last row is the one that catches people. Fixing the error by enlarging the window can quietly replace a loud failure with a silent one.

#When none of them are enough

If a single indivisible document genuinely exceeds the largest window you can afford, the shape of the problem has changed: you need retrieval over that document rather than one pass through it. Chunk it, index it, and ask targeted questions. That's more work than any fix above, which is why it comes last.

#What you should check before you ship the fix

Once you're under the limit, run 20 of your real inputs through and read the answers rather than the status codes. Context errors usually hide a second problem — your retrieval is pulling 15 chunks where 6 would do, or your agent has been resending 40 turns of history since the day you built it. The error was the only thing telling you. If your answers were already vague before the error appeared, trimming won't help. Look at how your chunks are being ranked instead.

Related reading: how to fix OpenAI rate limit errors, which the same workloads tend to trigger, and cutting your LLM bill.

Frequently asked questions

Do images count toward the context limit?

Yes. Vision inputs are tokenised and can be expensive, often several hundred tokens per image depending on resolution.

Does the system prompt count?

Every message counts, including the system prompt and every tool definition you pass with the request.

Is a bigger context window always better?

No. Retrieval accuracy drops in the middle of very long contexts, so a well-trimmed short context often outperforms a stuffed long one.

How do I count tokens without calling the API?

Use tiktoken locally for OpenAI models. It runs offline and matches the server's count closely enough to budget against.

Written by

Vemuri Abhiram

Founder & Editor

I write every post here. The tools I review are ones I pay for and run on real client work, not demo accounts, and every number you see came off my own invoice or dashboard.

More from Underread, once a week.

The guides worth reading and the tests worth trusting. One email, every Friday.

No spam. Unsubscribe in one click.