Skip to content
Underread.
Guides

How to Fix “Rate limit reached for requests” on the OpenAI API (6 Fixes)

Six fixes, in the order I would try them. The first one tells you which limit you actually hit, and most people are tuning the wrong one.

By Vemuri Abhiram6 min read
Developer workstation with code open on two monitors
Developer workstation with code open on two monitors
On this page

Jobs that ran fine yesterday start failing. Every third call comes back red, the queue backs up, and nothing in your code changed:

openai.RateLimitError: Rate limit reached for requests

Your traffic shape changed, even if your code didn't. More users, a longer prompt, a retry loop that got enthusiastic.

There are 6 fixes below, ordered from fastest to most involved. Stop at the one that fixes it.

#The fix order at a glance

# Fix Time Fixes which limit
1 Read the rate-limit headers 2 min Tells you which
2 Exponential backoff with jitter 15 min RPM, TPM
3 Batch small calls together 1 hour RPM
4 Cut tokens per call 1 hour TPM
5 Move scheduled work to the Batch API Half a day All three
6 Request a higher usage tier Days All three

#1. Read the headers before you change anything

Skip this and you can lose an afternoon tuning the wrong number. I've watched a team spend a day adding delays between calls when their problem was token volume, where delays do nothing at all.

Every response carries your current standing. Log these four:

x-ratelimit-remaining-requests
x-ratelimit-remaining-tokens
x-ratelimit-reset-requests
x-ratelimit-reset-tokens

Whichever remaining value reaches zero is the limit you're hitting.

  • Requests drop to zero — you're making too many calls. Batch them.
  • Tokens drop to zero — your calls are too big. Trim prompts.

The reset values are how long until the window refills, expressed like 6m0s. That number tells you how long a backoff actually needs to wait.

#2. Add real backoff, with jitter

A fixed sleep(1) in a retry loop is why the failures cluster. 10 workers fail at the same moment, all sleep one second, and all retry on the same tick. Jitter (a small random delay added to each wait) is the part people leave out, and it's the part that spreads the retry storm:

import random, time
from openai import OpenAI, RateLimitError

client = OpenAI()

def call(**kw):
    for attempt in range(6):
        try:
            return client.chat.completions.create(**kw)
        except RateLimitError:
            if attempt == 5:
                raise
            # 1s, 2s, 4s, 8s, 16s — plus up to a second of jitter so that
            # concurrent workers do not retry in lockstep.
            time.sleep((2 ** attempt) + random.random())

6 attempts with this schedule covers about half a minute of backoff, which clears most RPM windows.

#3. Batch the small calls

If you classify 500 rows with one call each, you spend 500 requests against a per-minute request limit to move a few thousand tokens. The work batches cleanly:

Approach Requests RPM used Notes
One row per call 500 500 Hits RPM immediately
25 rows per call 20 20 Same tokens, 4% of the requests
Batch API 1 submission 0 Off synchronous limits entirely

25 items per call is a reasonable starting point for short classification work. Push it until output quality drops, then step back one notch.

#4. Cut tokens, not just calls

If the headers point at tokens, no amount of spacing calls out will help. The volume is the problem.

  • Trim the system prompt. A 900-token preamble sent 400 times is 360,000 tokens of pure overhead before any real work happens.
  • Cap max_tokens. An uncapped completion reserves its maximum against your limit even when the actual reply is 2 sentences.
  • Stop resending history. Most agent frameworks resend the entire message list every turn by default. By turn 15 you're paying to transmit turns one to fourteen again.

#5. Move scheduled work off the synchronous path

Anything that doesn't need an answer in the next few seconds — nightly enrichment, classifying yesterday's tickets, backfills — has no reason to compete with your live traffic for the same limit. The Batch API takes that work off your RPM and TPM entirely, and costs less, in exchange for a turnaround measured in hours.

This is the single biggest structural fix on the list. It's also the only one that keeps paying off as you grow.

#6. Ask for a higher tier, last

Usage tiers move on spend and account age. Before you write in, do the 5 things above, because the first question back is always whether you're batching.

#Match your symptom to the fix

What you're seeing Most likely cause Go to
Fails under load, fine when quiet RPM, too many small calls Fixes 2 and 3
Fails on long documents only TPM, prompts too large Fix 4
Fails at the same time each night A batch job competing with live traffic Fix 5
Fails from minute one, every day RPD, or a tier too low Fix 6
Errors cluster in bursts No jitter in the retry loop Fix 2

#When all 6 fail

If you're batching, capped, backing off and still limited, you've outgrown the tier rather than misconfigured it. At that point the useful questions are whether a smaller model handles part of the workload, and whether any of the traffic can be served from a cache of previous answers. Both are real engineering, which is why they come after the 6 above.

#A sane default setup

Backoff with jitter, a hard max_tokens, batching for anything over 20 items, scheduled work on the Batch API, and the rate-limit headers logged wherever you already send metrics. That combination keeps most small teams off the limit permanently, and it tells you immediately which limit is moving when you do hit one.

#What this costs you if you leave it

Here's the part that's easy to miss: you aren't just losing the 3 or 4 calls that came back red. A queue that backs up behind a 60-second window will usually drop work you never see. If your retries aren't idempotent (safe to run twice without doing the thing twice) you can end up charging a customer twice for one job. I watched a team lose 2 days of enrichment that way, and their dashboard showed 200 successes, because the failures never reached it.

Related reading: cutting your LLM bill without changing models, and fixing context length errors, which often shows up in the same week.

Frequently asked questions

Does upgrading my plan remove the limit?

No, it raises it. Limits exist at every tier including the highest, so backoff isn't optional at any level of spend.

Why do I hit it with so few requests?

Almost always the token-per-minute limit rather than requests. Check x-ratelimit-remaining-tokens before assuming you're making too many calls.

Is the Batch API cheaper as well as off-limit?

Yes, materially cheaper, in exchange for a turnaround measured in hours rather than seconds.

How many retries should I allow?

6 attempts with exponential backoff covers roughly 30 seconds, which clears most per-minute windows. More than that and you're queueing work that should go to batch.

Does streaming change anything?

No. A streamed response counts the same tokens against the same limit; you just see the first ones sooner.

Written by

Vemuri Abhiram

Founder & Editor

I write every post here. The tools I review are ones I pay for and run on real client work, not demo accounts, and every number you see came off my own invoice or dashboard.

More from Underread, once a week.

The guides worth reading and the tests worth trusting. One email, every Friday.

No spam. Unsubscribe in one click.