Scraping and crawling for AI agents

7 APIs compared on price, free tier, MCP support and real usage. Prices from the vendors' pricing pages, checked Oct 9, 2026.

Cheapest per 1,000 requests

$0.5 per 1k at list price

Most installed SDK

2.5M downloads a week

Free to start

7 of 7

Firecrawl, Apify, Crawl4AI

Official MCP server

7 of 7

plug straight into Claude Code, Cursor and others

API	Free	Price	SDK downloads / wk	GitHub stars	MCP
Firecrawl Scrapes, crawls, maps and searches websites, returning clean markdown or JSON for LLMs.	1,000 credits/month, no credit card (500...	$5 / 1k requests · free tier	2.5M	190K	MCP
Apify Platform of ready-made scrapers (Actors) plus serverless runtime and proxies for web data.	$5/month of platform usage, no credit card	Free plan · paid from $19/mo	1M	99	MCP
Crawl4AI Open-source LLM-friendly crawler you can self-host, now also offered as a hosted cloud API.	Self-host free (open source); Cloud Free...	$0.5 / 1k requests · free tier	292K	85.1K	MCP
Spider Fast crawling and scraping API built on the open-source Rust spider engine, streaming markdown or JSON.	New accounts start with a free balance...	Free plan · paid from $40/mo	110K	2,766	MCP
ScrapingBee Web scraping API that handles headless browsers, proxies and anti-bot, returning HTML, markdown or JSON.	1,000 free API credits, no credit card	Free plan · paid from $49/mo	88.9K	132	MCP
Bright Data (Unlocker API) Unblocking scraping API, SERP API, scraping browser and proxy network for large-scale web data.	5K Unlocker requests/month, no credit card	$1.5 / 1k requests · free tier	28K	2,664	MCP
Jina Reader (r.jina.ai) Converts any URL (including PDFs) into clean LLM-ready markdown by prefixing r.jina.ai.	Free without a key at 20 RPM; 10M free...	Free plan	—	12.1K	MCP

Other categories