How to Fix “Rate limit reached for requests” on the OpenAI API (6 Fixes)
Six fixes, in the order I would try them. The first one tells you which limit you actually hit, and most people are tuning the wrong one.
On this page
- The fix order at a glance
- 1. Read the headers before you change anything
- 2. Add real backoff, with jitter
- 3. Batch the small calls
- 4. Cut tokens, not just calls
- 5. Move scheduled work off the synchronous path
- 6. Ask for a higher tier, last
- Match your symptom to the fix
- When all 6 fail
- A sane default setup
- What this costs you if you leave it
Jobs that ran fine yesterday start failing. Every third call comes back red, the queue backs up, and nothing in your code changed:
openai.RateLimitError: Rate limit reached for requests
Your traffic shape changed, even if your code didn't. More users, a longer prompt, a retry loop that got enthusiastic.
There are 6 fixes below, ordered from fastest to most involved. Stop at the one that fixes it.
#The fix order at a glance
| # | Fix | Time | Fixes which limit |
|---|---|---|---|
| 1 | Read the rate-limit headers | 2 min | Tells you which |
| 2 | Exponential backoff with jitter | 15 min | RPM, TPM |
| 3 | Batch small calls together | 1 hour | RPM |
| 4 | Cut tokens per call | 1 hour | TPM |
| 5 | Move scheduled work to the Batch API | Half a day | All three |
| 6 | Request a higher usage tier | Days | All three |
#1. Read the headers before you change anything
Skip this and you can lose an afternoon tuning the wrong number. I've watched a team spend a day adding delays between calls when their problem was token volume, where delays do nothing at all.
Every response carries your current standing. Log these four:
x-ratelimit-remaining-requests
x-ratelimit-remaining-tokens
x-ratelimit-reset-requests
x-ratelimit-reset-tokens
Whichever remaining value reaches zero is the limit you're hitting.
- Requests drop to zero — you're making too many calls. Batch them.
- Tokens drop to zero — your calls are too big. Trim prompts.
The reset values are how long until the window refills, expressed like 6m0s. That number tells you how long a backoff actually needs to wait.
#2. Add real backoff, with jitter
A fixed sleep(1) in a retry loop is why the failures cluster. 10 workers fail at the same moment, all sleep one second, and all retry on the same tick. Jitter (a small random delay added to each wait) is the part people leave out, and it's the part that spreads the retry storm:
import random, time
from openai import OpenAI, RateLimitError
client = OpenAI()
def call(**kw):
for attempt in range(6):
try:
return client.chat.completions.create(**kw)
except RateLimitError:
if attempt == 5:
raise
# 1s, 2s, 4s, 8s, 16s — plus up to a second of jitter so that
# concurrent workers do not retry in lockstep.
time.sleep((2 ** attempt) + random.random())
6 attempts with this schedule covers about half a minute of backoff, which clears most RPM windows.
#3. Batch the small calls
If you classify 500 rows with one call each, you spend 500 requests against a per-minute request limit to move a few thousand tokens. The work batches cleanly:
| Approach | Requests | RPM used | Notes |
|---|---|---|---|
| One row per call | 500 | 500 | Hits RPM immediately |
| 25 rows per call | 20 | 20 | Same tokens, 4% of the requests |
| Batch API | 1 submission | 0 | Off synchronous limits entirely |
25 items per call is a reasonable starting point for short classification work. Push it until output quality drops, then step back one notch.
#4. Cut tokens, not just calls
If the headers point at tokens, no amount of spacing calls out will help. The volume is the problem.
- Trim the system prompt. A 900-token preamble sent 400 times is 360,000 tokens of pure overhead before any real work happens.
- Cap
max_tokens. An uncapped completion reserves its maximum against your limit even when the actual reply is 2 sentences. - Stop resending history. Most agent frameworks resend the entire message list every turn by default. By turn 15 you're paying to transmit turns one to fourteen again.
#5. Move scheduled work off the synchronous path
Anything that doesn't need an answer in the next few seconds — nightly enrichment, classifying yesterday's tickets, backfills — has no reason to compete with your live traffic for the same limit. The Batch API takes that work off your RPM and TPM entirely, and costs less, in exchange for a turnaround measured in hours.
This is the single biggest structural fix on the list. It's also the only one that keeps paying off as you grow.
#6. Ask for a higher tier, last
Usage tiers move on spend and account age. Before you write in, do the 5 things above, because the first question back is always whether you're batching.
#Match your symptom to the fix
| What you're seeing | Most likely cause | Go to |
|---|---|---|
| Fails under load, fine when quiet | RPM, too many small calls | Fixes 2 and 3 |
| Fails on long documents only | TPM, prompts too large | Fix 4 |
| Fails at the same time each night | A batch job competing with live traffic | Fix 5 |
| Fails from minute one, every day | RPD, or a tier too low | Fix 6 |
| Errors cluster in bursts | No jitter in the retry loop | Fix 2 |
#When all 6 fail
If you're batching, capped, backing off and still limited, you've outgrown the tier rather than misconfigured it. At that point the useful questions are whether a smaller model handles part of the workload, and whether any of the traffic can be served from a cache of previous answers. Both are real engineering, which is why they come after the 6 above.
#A sane default setup
Backoff with jitter, a hard max_tokens, batching for anything over 20 items, scheduled work on the Batch API, and the rate-limit headers logged wherever you already send metrics. That combination keeps most small teams off the limit permanently, and it tells you immediately which limit is moving when you do hit one.
#What this costs you if you leave it
Here's the part that's easy to miss: you aren't just losing the 3 or 4 calls that came back red. A queue that backs up behind a 60-second window will usually drop work you never see. If your retries aren't idempotent (safe to run twice without doing the thing twice) you can end up charging a customer twice for one job. I watched a team lose 2 days of enrichment that way, and their dashboard showed 200 successes, because the failures never reached it.
Related reading: cutting your LLM bill without changing models, and fixing context length errors, which often shows up in the same week.
Frequently asked questions
Does upgrading my plan remove the limit?
No, it raises it. Limits exist at every tier including the highest, so backoff isn't optional at any level of spend.
Why do I hit it with so few requests?
Almost always the token-per-minute limit rather than requests. Check x-ratelimit-remaining-tokens before assuming you're making too many calls.
Is the Batch API cheaper as well as off-limit?
Yes, materially cheaper, in exchange for a turnaround measured in hours rather than seconds.
How many retries should I allow?
6 attempts with exponential backoff covers roughly 30 seconds, which clears most per-minute windows. More than that and you're queueing work that should go to batch.
Does streaming change anything?
No. A streamed response counts the same tokens against the same limit; you just see the first ones sooner.
Written by
Vemuri Abhiram
Founder & Editor
I write every post here. The tools I review are ones I pay for and run on real client work, not demo accounts, and every number you see came off my own invoice or dashboard.
You might also like
How to Fix “This model's maximum context length is…” (4 Fixes)
The context length error explained, plus four fixes ranked by how much answer quality each one costs — and why a bigger window should be last.
Vemuri Abhiram5 min read
How to Cut Your LLM API Bill Without Changing Models (6 Changes)
Six ways to cut an LLM API bill that do not involve downgrading the model, ordered by how much they typically save.
Vemuri Abhiram5 min read
OpenAI Cuts API Prices Again: What Changes for Small Builds
What another round of API price cuts actually changes for small teams, and the two changes that matter more than the headline rate.
Vemuri Abhiram2 min read
More from Underread, once a week.
The guides worth reading and the tests worth trusting. One email, every Friday.
No spam. Unsubscribe in one click.