Quick answer
How many GB of proxy traffic do I need to scrape 1,000 pages?
About 0.05 GB for 1,000 compressed HTML pages, about 0.5 GB with a headless browser that blocks images, and about 2.2 GB for 1,000 fully rendered pages, using the HTTP Archive Web Almanac 2025 median of 2,164 KB per mobile page. Multiply by your price per GB: CrawlRelay residential traffic costs $0.50/GB from 50 GB and never expires. No KYC. Pay in crypto.
Traffic math at a glance
| Formula | GB = pages × average compressed KB per page ÷ 1,000,000 |
|---|---|
| Median page weight, all resources | 2,164 KB on mobile, 2,412 KB on desktop (Web Almanac 2025, July 2025 crawl) |
| Median HTML | 22 KB on home pages; 33–35 KB in another measure of the same chapter |
| Units | 1 GB = 1,000,000,000 bytes (decimal, the unit CrawlRelay bills in); 1 GiB = 1,073,741,824 bytes |
| Biggest saving | Fetch HTML or JSON instead of rendering; block images, fonts and media when you render |
| CrawlRelay price | $1.50 for 1 GB; $0.50/GB from 50 GB |
How many GB and dollars per 1,000 pages?
Typical compressed sizes per page type. The three cost columns use reference prices per GB so you can read off any provider; CrawlRelay's own price is $0.50/GB from 50 GB.
| What you fetch | ≈ KB per page | GB per 1,000 pages | At $0.50/GB | At $1.00/GB | At $3.50/GB |
|---|---|---|---|---|---|
| JSON API response | 10 | 0.01 | $0.005 | $0.01 | $0.035 |
| HTML page, compressed | 50 | 0.05 | $0.025 | $0.05 | $0.18 |
| Heavy HTML (large inline scripts) | 150 | 0.15 | $0.075 | $0.15 | $0.53 |
| Headless browser, images, fonts and media blocked | 500 | 0.5 | $0.25 | $0.50 | $1.75 |
| Full render, median mobile page | 2,164 | 2.16 | $1.08 | $2.16 | $7.57 |
$3.50/GB is the median list price at 10 GB in our October 2026 price index. Page weights from the HTTP Archive Web Almanac 2025 and typical compressed sizes; measure your own pages.
How do I measure my own pages?
Fetch a sample of 100 to 1,000 real URLs through the proxy, add up the bytes received on the wire, and extrapolate. httpx's num_bytes_downloaded counts compressed bytes, which is close to what a proxy bills. Add a few kilobytes per new connection for the TLS handshake if you rotate the IP on every request.
import statistics
import httpx
PROXY = "http://USERNAME-country-us:[email protected]:7777"
SAMPLE = ["https://example.com/", "https://example.org/", "https://example.net/"] # use 100+ real URLs
sizes = []
with httpx.Client(proxy=PROXY, timeout=httpx.Timeout(30.0, connect=10.0), follow_redirects=True) as client:
for url in SAMPLE:
try:
r = client.get(url)
sizes.append(r.num_bytes_downloaded)
except httpx.HTTPError:
pass
if sizes:
mean = statistics.mean(sizes)
print("pages measured:", len(sizes))
print("mean bytes per page: %.0f" % mean)
print("GB per 1,000 pages: %.3f" % (mean * 1000 / 1e9))
print("GB for 1 million pages: %.1f" % (mean * 1_000_000 / 1e9))Which plan fits which project?
Turn your page count into gigabytes, then pick the plan that covers it. With HTML at about 50 KB per page:
| Project | Pages | Traffic | CrawlRelay plan |
|---|---|---|---|
| Test a new target | 5,000 | ≈0.25 GB | 1 GB ($1.50) |
| Small monitor, one month | 100,000 | ≈5 GB | 5 GB ($3.75) or 10 GB ($5.94) |
| Catalogue crawl | 1 million | ≈50 GB | 50 GB ($25) |
| Multi-site corpus | 10 million | ≈500 GB | 500 GB ($250) |
| Large AI corpus | 20 million | ≈1 TB | 1 TB ($500) |
Traffic never expires, so you can buy the larger plan once and use it over several months.
How do I cut proxy traffic without losing data?
- Fetch the data, not the page. If a site documents or allows its JSON endpoints, they are a fraction of the HTML.
- Render only when you must. A headless browser fetches scripts, styles, images and fonts; plain HTTP fetches one document.
- Block heavy resources in browsers: images, fonts and media are most of a page's bytes.
- Keep compression on. Send
Accept-Encoding: gzip, br; most clients do by default. - Skip unchanged pages with
If-None-MatchorIf-Modified-Since: a304costs a few hundred bytes. - Deduplicate URLs before you queue them: tracking parameters and session IDs create copies of the same page.
- Store raw responses so a parser bug never forces a second download.
Proxy bandwidth: questions
How much does it cost to scrape 1 million pages with residential proxies?
With compressed HTML at about 50 KB per page, 1 million pages use about 50 GB. At CrawlRelay that is the 50 GB plan, $25, as of October 2026. Rendering every page in a browser at the median 2 MB would use about 2 TB instead, so the fetch method matters forty times more than the provider.
Do proxies count upload and download traffic?
Proxy bandwidth counts the data that passes through the gateway for your connections; for scraping, responses are almost all of it. Requests with large bodies, such as file uploads, add their own size. At CrawlRelay: Usage counts all data sent and received through the proxy (upload and download, headers included). 1 GB = 1,000,000,000 bytes. Connection attempts that fail before reaching the target are not counted.
Does a new IP per request use more traffic?
Slightly. Each new connection needs a TLS handshake with the website, typically a few kilobytes including the certificate chain. On 50 KB pages that is a few percent; on small JSON responses it can be a large share. Use sticky sessions with keep-alive for many requests to one site when IP diversity does not matter.
Is the price per GB or the GB per page more important?
GB per page, by far. The spread between fetching HTML and rendering full pages is about forty-fold, while median list prices are only two to five times the cheapest ones at the same volume. Optimise the fetch method first, then compare providers on the price table.
Sources (2)
- Web Almanac 2025: Page Weight — HTTP Archive (2026-01-15)
- Residential proxy price index 2026 — CrawlRelay (2026-10-13)
Buy the gigabytes you measured
$1.50 for 1 GB, $0.50/GB from 50 GB, never expiring.