# 403 Forbidden when scraping through a proxy: causes and what to do > Why a site answers 403 Forbidden to a scraper behind a proxy, how to tell a block from a bug, and what to do without evading the site's rules. - URL: https://crawlrelay.com/docs/errors/403-forbidden/ - Last updated: 2026-10-13 - Publisher: CrawlRelay (https://crawlrelay.com) HTTP status code (4xx, client error): the exact message in each tool, the likely causes in order, and the fix that works. **Quick answer — What does 403 Forbidden mean with a proxy?** **A 403 Forbidden** means the server understood your request and refuses to serve it. When you scrape through CrawlRelay residential proxies, a 403 almost always comes from the target site or its firewall, not from the proxy. Find out why before you send more traffic: a wrong country, a missing header, a page behind a login, or a site that does not allow automated access. ## 403 Forbidden at a glance | Attribute | Value | | --- | --- | | Error | 403 Forbidden | | Type | HTTP status code (4xx, client error) | | Sent by | Usually the target website or its firewall; sometimes the proxy for a blocked destination | | Defined in | RFC 9110, section 15.5.4 | | Retry it? | Once, after a pause and on a new IP; repeated 403s mean stop | | First check | Open the same URL in a normal browser from the same country | ## What does 403 Forbidden mean? RFC 9110 defines 403 as a refusal: the server understood the request and will not fulfil it. Unlike 401, new credentials will not help. A site may explain the reason in the response body, but many firewalls return a short generic page. For a scraper, a 403 is a policy signal. The site has decided that this request, from this IP, with these headers, at this rate, should not get the page. Some of those inputs are yours to fix. Others are the site telling you no. ## What does a 403 look like in Python, curl, Node.js and browsers? The same failure prints differently in every tool. These are the messages to search your logs for: - **Python requests:** `r.status_code == 403`, often with an HTML body from a firewall vendor - **Scrapy:** `Ignoring response <403 https://...>: HTTP status code is not handled or not allowed` - **curl:** `HTTP/2 403` in the verbose output - **Playwright, Puppeteer:** the navigation succeeds but `response.status()` returns 403 and the page shows an access-denied message ## What causes a 403? Most likely first: - **Geography.** The content is not offered in the exit IP's country, or the site serves only one region. - **IP reputation.** The site blocks address ranges it associates with automated traffic. Datacenter ranges are blocked more often than residential ones. - **Request fingerprint.** Missing or inconsistent headers, such as no `Accept-Language`, or a browser User-Agent sent by a plain HTTP client. - **Rate.** Too many requests in a short time from one session, so the firewall escalates from 429 to 403. - **Access rules.** The page needs a login, a paid account, or is excluded for crawlers in robots.txt or the terms of use. - **Proxy policy.** Some destinations are refused by the proxy itself. At CrawlRelay that covers the categories listed in the Acceptable Use Policy. ## How do I diagnose a 403 step by step? 1. Load the URL in a normal browser from the same country. If it fails there too, the page is not public for that location. 2. Compare the headers your scraper sends with the ones the browser sends. 3. Check the target's robots.txt and terms for rules on automated access. 4. Look at the time pattern: 403s that start after N requests point to rate, not to the IP. *Test the failing URL with curl (HTTP port, verbose)* ```bash # Use the URL that fails; with https://api.ipify.org instead, you test the gateway alone. # "HTTP/1.1 200 Connection established" first, then the site's own status: the site answered. curl -v -x http://USERNAME:PASSWORD@gw.crawlrelay.com:7777 https://TARGET_URL # Same test through the SOCKS5 port, with DNS resolved at the exit curl -v --socks5-hostname gw.crawlrelay.com:7778 --proxy-user USERNAME:PASSWORD https://TARGET_URL ``` ## How do I fix a 403? - **Pick the right country** with the `country-` parameter so you see the edition of the site that local visitors see. - **Send honest, complete headers.** Identify your crawler in the User-Agent with a contact URL, and send `Accept` and `Accept-Language`. - **Slow down.** Lower concurrency per host and add delays. See [how to handle 403, 429 and CAPTCHAs](https://crawlrelay.com/guides/403-429-captcha/). - **Respect a refusal.** If a site blocks automated access on purpose, look for an official API, a data licence or a public dataset instead. Rotating IPs to push past a deliberate block is outside our [Acceptable Use Policy](https://crawlrelay.com/acceptable-use/). ## Should I retry after a 403? Retry a 403 at most once, after a pause, on a new IP. One retry separates a single flagged address from a site-wide rule. If the second attempt also returns 403, stop sending requests to that URL pattern and review your setup. Hammering a 403 wastes bandwidth and turns a soft block into a hard one. ## How does 403 Forbidden apply to CrawlRelay? On CrawlRelay a new connection without a session ID usually exits through a different residential IP, so a single retry normally tests a fresh address. Keep the same country when you retry: switching countries changes the content you compare. Requests to banking, government and payment-portal sites are refused by the gateway under the AUP. Gateway: `gw.crawlrelay.com`, port 7777 (HTTP, HTTPS through CONNECT) and port 7778 (SOCKS5). Username format: `USERNAME-country-{cc}-state-{state}-city-{city}-zip-{zip}-asn-{asn}-session-{id}-ttl-{minutes}`. Related: [429 Too Many Requests](https://crawlrelay.com/docs/errors/429-too-many-requests/) · [401 vs 407](https://crawlrelay.com/docs/errors/401-unauthorized/) · [Handling 403, 429 and CAPTCHAs](https://crawlrelay.com/guides/403-429-captcha/). ## 403 Forbidden: questions developers ask ### Does a 403 mean my proxy IP is banned? Not necessarily. A 403 can come from the wrong country, missing headers, request rate or a page that is not public. Test the same URL from a browser in the same country and compare. If a fresh IP gets the page and the old one does not, the old IP was flagged. If every IP gets 403, the site's rules are the cause. ### Will residential proxies remove 403 errors? Not as a rule. A residential exit in the right country or city fixes a 403 caused by the wrong location, because the site then serves its local edition. If a site filters hosting ranges or refuses crawlers to keep automated traffic out, a residential IP does not make that access permitted. 403s caused by headers, rate or login walls need fixes in your code. ### Is it legal to keep scraping after a 403? A 403 is a clear signal that the site refuses the request. Pushing past deliberate technical barriers can create legal risk in several countries, for example under anti-circumvention or computer-misuse laws. This is not legal advice. Read our overview of [web scraping law in 2026](https://crawlrelay.com/guides/is-web-scraping-legal/) and ask counsel for your case. ## Other proxy errors in this reference - [401 Unauthorized vs 407](https://crawlrelay.com/docs/errors/401-unauthorized/) - [403 Forbidden](https://crawlrelay.com/docs/errors/403-forbidden/) - [407 Proxy Authentication Required](https://crawlrelay.com/docs/errors/407-proxy-authentication-required/) - [429 Too Many Requests](https://crawlrelay.com/docs/errors/429-too-many-requests/) - [502 Bad Gateway](https://crawlrelay.com/docs/errors/502-bad-gateway/) - [503 Service Unavailable](https://crawlrelay.com/docs/errors/503-service-unavailable/) - [504 Gateway Timeout](https://crawlrelay.com/docs/errors/504-gateway-timeout/) - [Cannot connect to proxy (ProxyError)](https://crawlrelay.com/docs/errors/cannot-connect-to-proxy/) - [Connection reset by peer (ECONNRESET)](https://crawlrelay.com/docs/errors/connection-reset/) - [ERR_PROXY_CONNECTION_FAILED](https://crawlrelay.com/docs/errors/err-proxy-connection-failed/) - [ERR_TUNNEL_CONNECTION_FAILED](https://crawlrelay.com/docs/errors/err-tunnel-connection-failed/) - [Read timed out (ReadTimeout)](https://crawlrelay.com/docs/errors/read-timeout/) - [SOCKS5 proxy errors](https://crawlrelay.com/docs/errors/socks5-proxy-errors/) - [SSL: CERTIFICATE_VERIFY_FAILED](https://crawlrelay.com/docs/errors/ssl-certificate-verify-failed/) - [SSL: WRONG_VERSION_NUMBER](https://crawlrelay.com/docs/errors/ssl-wrong-version-number/) **Residential proxies with docs that cover the failure cases** CrawlRelay residential proxies from $0.50/GB, traffic that never expires. [See pricing](https://crawlrelay.com/pricing/) · [Quick start](https://crawlrelay.com/docs/) --- CrawlRelay: CrawlRelay is a developer-first rotating residential and 4G/5G mobile proxy service for scrapers and AI data pipelines: non-expiring pay-as-you-go traffic from $0.50/GB, paid in crypto, with no ID upload. No KYC — no ID, no selfie, no documents. Pay in crypto.