AI training data
Multi-terabyte public-web corpora: about 50 GB per million HTML pages, robots.txt and AI opt-outs, personal-data filtering.
Quick answer
What are residential proxies used for by developers?
Developers use CrawlRelay residential proxies to collect public web data from the location that matters: AI training corpora, page fetching for AI agents, location-accurate checks of search and ad pages, and e-commerce price monitoring. Traffic costs $1.50 for 1 GB and $0.50/GB from 50 GB, as of October 2026, and never expires. Paid in crypto, with no KYC.
Use cases
AI agents
A safe fetch tool and an MCP server through residential exits, with per-task sessions and guard rails.
SERP and rank checks
What still works in 2026: Google's robots.txt, the lawsuits, the blocks, and city-accurate checks of your own pages.
Price monitoring
Local prices, stock and delivery terms per country, JSON-LD extraction, traffic per product per month.
Which setup fits each use case?
| Use case | Proxy type | Sessions | Typical traffic |
|---|---|---|---|
| AI training data | Residential | Rotate; sticky only for pagination | 30–60 GB per million HTML pages |
| AI agents | Residential; mobile for mobile-only views | One sticky session per task | ≈1 MB per 20-page task in HTML |
| SERP and rank checks | Residential with city targeting | One short session per city check | Small: a few pages per check |
| Price monitoring | Residential per market | Rotate between products; sticky inside a flow | ≈1.8 MB per product per month, daily checks |
What do all these use cases have in common?
Public data, collected at a pace the site accepts, from the place where the data differs. Every CrawlRelay account works under the same Acceptable Use Policy: no logins to accounts you do not own, no personal data collected in breach of the law, no attempts to defeat a site's deliberate blocks, and a gateway that refuses Banking, government and payment-portal sites, plus SMTP (email-sending) ports 25, 465 and 587.
The engineering habits are shared too: identify your crawler, cap concurrency per host, fetch HTML before you render, skip unchanged pages, and keep each pipeline on its own sub-user with a traffic cap. The docs cover each one with code.
Which use cases does CrawlRelay not serve?
Anything that depends on impersonating people or defeating protections: creating or taking over accounts, buying limited stock or tickets with bots, sending spam, credential testing, fraud, or collecting personal data behind logins. Those uses break our policy and are the reason residential networks were seized in 2026. We suspend accounts that use the gateway for them.
Use cases: questions
Can I use CrawlRelay for social media data?
Only for public pages, collected within the platform's rules and the law, and without logging in to accounts you do not own. Much social media content is personal data, so check the legal basis first. Many platforms offer research or business APIs; use them where they exist.
Can I use CrawlRelay for ad verification?
Yes. Checking how your ads and landing pages appear in each country and city is a standard use of residential exits. Use city targeting and a short sticky session per check, as on the SERP page.
Is there a use case that needs mobile proxies?
Checking what phone users see: mobile sites, apps and mobile ads. Rotating mobile proxies cost $3.18 for 1 GB and $1.50/GB from 5 GB at CrawlRelay; test them on the small plan before routing more traffic there.