Rate limits

Limits#

• Requests / minute per key

  Default

  60 rpm

  Scope

  Each API key

  Notes

  Set per key in API keys (the rpm field).

• Requests / minute per account

  Default

  300 rpm

  Scope

  All keys of the account combined

  Notes

  Prevents getting around the limit by creating more keys.

• Concurrent requests

  Default

  10

  Scope

  Each account

  Notes

  Requests in flight at the same time (including open streams).

Per-minute limits use a 60-second sliding window; they don’t reset on the clock minute. Administrators can change the defaults; if you need higher limits for an enterprise account, contact support.

When you exceed a limit#

The gateway returns 429 rate_limit_exceeded with a Retry-After header (whole seconds):

HTTP/1.1 429 Too Many Requests
Retry-After: 7
X-Request-Id: req_3f0c…

{"error": {"message": "rate limit exceeded for this API key (60 requests per minute); retry after 7s",
           "type": "rate_limit_error", "code": "rate_limit_exceeded"}}

• rpm exceeded: Retry-After = seconds until the oldest request leaves the 60-second window.
• Concurrency exceeded: Retry-After: 1 — retry when another request finishes.
• 429 upstream_rate_limited means the upstream provider is rate limiting (not your own limits); handle it with backoff too.

Retrying correctly#

By default the OpenAI SDK retries 429s on its own (twice, with backoff, honouring Retry-After). If you write your own loop, respect Retry-After:

import time
from openai import OpenAI, RateLimitError

client = OpenAI(base_url="https://<host>/v1", api_key="sk-kompa-YOUR_API_KEY", max_retries=0)

def chat(messages, attempts=5):
    for i in range(attempts):
        try:
            return client.chat.completions.create(model="gpt-4o-mini", messages=messages)
        except RateLimitError as e:
            wait = int(e.response.headers.get("retry-after", 2 ** i))
            time.sleep(wait)
    raise RuntimeError("still rate limited")