Proxy error reference · HTTP status codes

503 Service Unavailable when scraping: what it means and how to fix it

HTTP status code (5xx, server error): the exact message in each tool, the likely causes in order, and the fix that works.

Last updated: October 13, 2026 · 5 min read

Quick answer

What does 503 Service Unavailable mean with a proxy?

503 Service Unavailable means the server cannot handle the request right now, because of overload, maintenance or a protective rule. Through CrawlRelay residential proxies, read the Retry-After header and the response body: a maintenance or overload 503 clears with time, while a challenge page means the site is filtering automated traffic and you should slow down.

503 Service Unavailable at a glance

Error	503 Service Unavailable
Type	HTTP status code (5xx, server error)
Sent by	The target website, its CDN, or the proxy when it has no exit to offer
Defined in	RFC 9110, section 15.6.4 (Retry-After: section 10.2.3)
Retry it?	Yes, after Retry-After or a backoff
First check	Check for a Retry-After header and whether the body is a CDN challenge page

What does 503 Service Unavailable mean?

RFC 9110 defines 503 as a temporary condition. The server is up but unwilling or unable to serve this request now, and it may say when to come back with Retry-After.

In practice, three different situations share this code: a site that is really overloaded or in maintenance, a CDN that shows a challenge page to traffic it considers automated, and a proxy gateway that has no exit available for very narrow targeting. The body and headers tell them apart.

What does a 503 look like in Python, curl, Node.js and browsers?

The same failure prints differently in every tool. These are the messages to search your logs for:

Python requests / httpx: status_code == 503, sometimes with Retry-After
Scrapy: retried by default (503 is in RETRY_HTTP_CODES)
curl: HTTP/2 503, often with a server: header naming the CDN
Browsers: a maintenance page or an interstitial check page

What causes a 503?

Most likely first:

Overload or maintenance on the target side, often with Retry-After.
A challenge page that a CDN serves to clients it wants to check, with status 503 or 403.
Your own load, when your crawler is a big share of the site's traffic.
Narrow targeting, such as a small city, when no exit matches at that moment.
An upstream outage at the CDN or cloud region the site uses.

How do I diagnose a 503 step by step?

Read Retry-After and the server header, then look at the HTML title of the body.
Load the page in a browser from the same country to see what a person sees.
Pause your crawler for a few minutes and test one request: if it passes, your load was the trigger.
Widen targeting to the country level and compare.

Test the failing URL with curl (HTTP port, verbose)

# Use the URL that fails; with https://api.ipify.org instead, you test the gateway alone.
# "HTTP/1.1 200 Connection established" first, then the site's own status: the site answered.
curl -v -x http://USERNAME:[email protected]:7777 https://TARGET_URL

# Same test through the SOCKS5 port, with DNS resolved at the exit
curl -v --socks5-hostname gw.crawlrelay.com:7778 --proxy-user USERNAME:PASSWORD https://TARGET_URL

How do I fix a 503?

Wait and retry with the Retry-After delay or an exponential backoff.
Cut your load on that host: fewer parallel requests, longer delays, caching.
Treat a challenge page as a stop signal for that pace. Do not use CAPTCHA-solving or browser-disguise tools to get through it; reduce the rate and contact the site if you need access.
Widen targeting when the 503 comes with a narrow city filter.

Python: retry with Retry-After, exponential backoff and jitter

import random
import time

import requests

PROXY = "http://USERNAME:[email protected]:7777"
PROXIES = {"http": PROXY, "https": PROXY}
RETRY_STATUSES = {429, 502, 503, 504}


def fetch(url, attempts=5, timeout=(10, 30)):
    for attempt in range(attempts):
        try:
            r = requests.get(url, proxies=PROXIES, timeout=timeout)
        except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
            r = None  # dropped exit, reset or timeout: a new connection gets a new exit
        if r is not None and r.status_code not in RETRY_STATUSES:
            return r
        retry_after = r.headers.get("Retry-After", "") if r is not None else ""
        if retry_after.isdigit():
            wait = int(retry_after)
        else:
            wait = min(60, 2 ** attempt) + random.uniform(0, 1)
        time.sleep(wait)
    raise RuntimeError("giving up on %s after %d attempts" % (url, attempts))

Should I retry after a 503?

Retry a 503, but patiently. Use Retry-After when present; otherwise back off exponentially from a few seconds up to a minute or two. If a host returns 503 to most requests for more than a few minutes, pause it entirely and come back later. Count challenge pages separately from real outages in your logs.

How does 503 Service Unavailable apply to CrawlRelay?

CrawlRelay sends your request out through a residential IP chosen to match your targeting. Country-level targeting draws on the widest set of exits; city-level targeting is narrower. If 503s appear only with a city filter, test the same request with the country only before you change anything else.

Gateway: gw.crawlrelay.com, port 7777 (HTTP, HTTPS through CONNECT) and port 7778 (SOCKS5). Username format: USERNAME-country-{cc}-state-{state}-city-{city}-zip-{zip}-asn-{asn}-session-{id}-ttl-{minutes}. Related: 429 Too Many Requests · 502 Bad Gateway · 504 Gateway Timeout.

503 Service Unavailable: questions developers ask

Why do I get 503 only through the proxy and not at home?

Either the site treats your proxied traffic differently, often showing a challenge page, or your targeting is narrow enough that exits are scarce. Compare the response body in both cases. A challenge page means slow down; a short proxy message with narrow targeting means widen the location filter.

Is a 503 the same as being blocked?

No. A real 503 is temporary and clears with time. A challenge page served with 503 is a filter, and it usually means your pace or pattern looks automated. Lower concurrency and add delays. If the site does not want automated access at all, respect that and look for an API or licensed data.

How many times should I retry a 503?

Three to five times with exponential backoff is a reasonable ceiling for one URL. Beyond that, pause the whole host instead of retrying single URLs, because a site in trouble does not recover faster under retries. Log the host and the time so you can resume later.

Other proxy errors in this reference

HTTP status codes

Browser and client errors

Connection and TLS errors

Residential proxies with docs that cover the failure cases

CrawlRelay residential proxies from $0.50/GB, traffic that never expires.

See pricing Quick start