# 503 Service Unavailable when scraping: what it means and how to fix it

> 503 Service Unavailable through a proxy: target overload, maintenance, a challenge page, or too narrow targeting. How to tell which one, and fix it.

- URL: https://crawlrelay.com/docs/errors/503-service-unavailable/
- Last updated: 2026-10-13
- Publisher: CrawlRelay (https://crawlrelay.com)

HTTP status code (5xx, server error): the exact message in each tool, the likely causes in order, and the fix that works.

**Quick answer — What does 503 Service Unavailable mean with a proxy?**

**503 Service Unavailable** means the server cannot handle the request right now, because of overload, maintenance or a protective rule. Through CrawlRelay residential proxies, read the `Retry-After` header and the response body: a maintenance or overload 503 clears with time, while a challenge page means the site is filtering automated traffic and you should slow down.

## 503 Service Unavailable at a glance

| Attribute | Value |
| --- | --- |
| Error | 503 Service Unavailable |
| Type | HTTP status code (5xx, server error) |
| Sent by | The target website, its CDN, or the proxy when it has no exit to offer |
| Defined in | RFC 9110, section 15.6.4 (Retry-After: section 10.2.3) |
| Retry it? | Yes, after Retry-After or a backoff |
| First check | Check for a Retry-After header and whether the body is a CDN challenge page |

## What does 503 Service Unavailable mean?

RFC 9110 defines 503 as a temporary condition. The server is up but unwilling or unable to serve this request now, and it may say when to come back with `Retry-After`.

In practice, three different situations share this code: a site that is really overloaded or in maintenance, a CDN that shows a challenge page to traffic it considers automated, and a proxy gateway that has no exit available for very narrow targeting. The body and headers tell them apart.

## What does a 503 look like in Python, curl, Node.js and browsers?

The same failure prints differently in every tool. These are the messages to search your logs for:

- **Python requests / httpx:** `status_code == 503`, sometimes with `Retry-After`
- **Scrapy:** retried by default (503 is in `RETRY_HTTP_CODES`)
- **curl:** `HTTP/2 503`, often with a `server:` header naming the CDN
- **Browsers:** a maintenance page or an interstitial check page

## What causes a 503?

Most likely first:

- **Overload or maintenance** on the target side, often with `Retry-After`.
- **A challenge page** that a CDN serves to clients it wants to check, with status 503 or 403.
- **Your own load**, when your crawler is a big share of the site's traffic.
- **Narrow targeting**, such as a small city, when no exit matches at that moment.
- **An upstream outage** at the CDN or cloud region the site uses.

## How do I diagnose a 503 step by step?

1. Read `Retry-After` and the `server` header, then look at the HTML title of the body.
2. Load the page in a browser from the same country to see what a person sees.
3. Pause your crawler for a few minutes and test one request: if it passes, your load was the trigger.
4. Widen targeting to the country level and compare.

*Test the failing URL with curl (HTTP port, verbose)*

```bash
# Use the URL that fails; with https://api.ipify.org instead, you test the gateway alone.
# "HTTP/1.1 200 Connection established" first, then the site's own status: the site answered.
curl -v -x http://USERNAME:PASSWORD@gw.crawlrelay.com:7777 https://TARGET_URL

# Same test through the SOCKS5 port, with DNS resolved at the exit
curl -v --socks5-hostname gw.crawlrelay.com:7778 --proxy-user USERNAME:PASSWORD https://TARGET_URL
```

## How do I fix a 503?

- **Wait and retry** with the `Retry-After` delay or an exponential backoff.
- **Cut your load** on that host: fewer parallel requests, longer delays, caching.
- **Treat a challenge page as a stop signal** for that pace. Do not use CAPTCHA-solving or browser-disguise tools to get through it; reduce the rate and contact the site if you need access.
- **Widen targeting** when the 503 comes with a narrow city filter.

*Python: retry with Retry-After, exponential backoff and jitter*

```python
import random
import time

import requests

PROXY = "http://USERNAME:PASSWORD@gw.crawlrelay.com:7777"
PROXIES = {"http": PROXY, "https": PROXY}
RETRY_STATUSES = {429, 502, 503, 504}


def fetch(url, attempts=5, timeout=(10, 30)):
    for attempt in range(attempts):
        try:
            r = requests.get(url, proxies=PROXIES, timeout=timeout)
        except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
            r = None  # dropped exit, reset or timeout: a new connection gets a new exit
        if r is not None and r.status_code not in RETRY_STATUSES:
            return r
        retry_after = r.headers.get("Retry-After", "") if r is not None else ""
        if retry_after.isdigit():
            wait = int(retry_after)
        else:
            wait = min(60, 2 ** attempt) + random.uniform(0, 1)
        time.sleep(wait)
    raise RuntimeError("giving up on %s after %d attempts" % (url, attempts))
```

## Should I retry after a 503?

Retry a 503, but patiently. Use `Retry-After` when present; otherwise back off exponentially from a few seconds up to a minute or two. If a host returns 503 to most requests for more than a few minutes, pause it entirely and come back later. Count challenge pages separately from real outages in your logs.

## How does 503 Service Unavailable apply to CrawlRelay?

CrawlRelay sends your request out through a residential IP chosen to match your targeting. Country-level targeting draws on the widest set of exits; city-level targeting is narrower. If 503s appear only with a city filter, test the same request with the country only before you change anything else.

Gateway: `gw.crawlrelay.com`, port 7777 (HTTP, HTTPS through CONNECT) and port 7778 (SOCKS5). Username format: `USERNAME-country-{cc}-state-{state}-city-{city}-zip-{zip}-asn-{asn}-session-{id}-ttl-{minutes}`. Related: [429 Too Many Requests](https://crawlrelay.com/docs/errors/429-too-many-requests/) · [502 Bad Gateway](https://crawlrelay.com/docs/errors/502-bad-gateway/) · [504 Gateway Timeout](https://crawlrelay.com/docs/errors/504-gateway-timeout/).

## 503 Service Unavailable: questions developers ask

### Why do I get 503 only through the proxy and not at home?

Either the site treats your proxied traffic differently, often showing a challenge page, or your targeting is narrow enough that exits are scarce. Compare the response body in both cases. A challenge page means slow down; a short proxy message with narrow targeting means widen the location filter.

### Is a 503 the same as being blocked?

No. A real 503 is temporary and clears with time. A challenge page served with 503 is a filter, and it usually means your pace or pattern looks automated. Lower concurrency and add delays. If the site does not want automated access at all, respect that and look for an API or licensed data.

### How many times should I retry a 503?

Three to five times with exponential backoff is a reasonable ceiling for one URL. Beyond that, pause the whole host instead of retrying single URLs, because a site in trouble does not recover faster under retries. Log the host and the time so you can resume later.

## Other proxy errors in this reference

- [401 Unauthorized vs 407](https://crawlrelay.com/docs/errors/401-unauthorized/)
- [403 Forbidden](https://crawlrelay.com/docs/errors/403-forbidden/)
- [407 Proxy Authentication Required](https://crawlrelay.com/docs/errors/407-proxy-authentication-required/)
- [429 Too Many Requests](https://crawlrelay.com/docs/errors/429-too-many-requests/)
- [502 Bad Gateway](https://crawlrelay.com/docs/errors/502-bad-gateway/)
- [503 Service Unavailable](https://crawlrelay.com/docs/errors/503-service-unavailable/)
- [504 Gateway Timeout](https://crawlrelay.com/docs/errors/504-gateway-timeout/)
- [Cannot connect to proxy (ProxyError)](https://crawlrelay.com/docs/errors/cannot-connect-to-proxy/)
- [Connection reset by peer (ECONNRESET)](https://crawlrelay.com/docs/errors/connection-reset/)
- [ERR_PROXY_CONNECTION_FAILED](https://crawlrelay.com/docs/errors/err-proxy-connection-failed/)
- [ERR_TUNNEL_CONNECTION_FAILED](https://crawlrelay.com/docs/errors/err-tunnel-connection-failed/)
- [Read timed out (ReadTimeout)](https://crawlrelay.com/docs/errors/read-timeout/)
- [SOCKS5 proxy errors](https://crawlrelay.com/docs/errors/socks5-proxy-errors/)
- [SSL: CERTIFICATE_VERIFY_FAILED](https://crawlrelay.com/docs/errors/ssl-certificate-verify-failed/)
- [SSL: WRONG_VERSION_NUMBER](https://crawlrelay.com/docs/errors/ssl-wrong-version-number/)

**Residential proxies with docs that cover the failure cases** CrawlRelay residential proxies from $0.50/GB, traffic that never expires. [See pricing](https://crawlrelay.com/pricing/) · [Quick start](https://crawlrelay.com/docs/)

---

CrawlRelay: CrawlRelay is a developer-first rotating residential and 4G/5G mobile proxy service for scrapers and AI data pipelines: non-expiring pay-as-you-go traffic from $0.50/GB, paid in crypto, with no ID upload. No KYC — no ID, no selfie, no documents. Pay in crypto.