Skip to content
Agents tracked: 492 Releases (7d): 545 Agent status: 1 with issuesSince last limit reset: Claude 9d · Codex 7d For AI agents: Markdown Updated Nov 20, 2026

AI agents for web scraping

Agents that open pages, click through them and return the data you asked for in a structured form.

By weekly package downloads, the leaders are Stagehand (2.9M), Browser Use (2.2M). By GitHub stars, the leaders are Browser Use (121K), Stagehand (26.1K), Skyvern (23.5K). Among agents with at least 20,000 downloads in the last 30 days, the fastest grower is Stagehand, with downloads up 40%. As of Nov 20, 2026.

Works with:
Usage
  1. 17
    Browser Use · Browser · browser automation library

    Open-source Python library and hosted cloud that let LLM agents control a real browser to complete web tasks, with a CLI, a cloud API for hosted agents and browsers, and its own BU 2.0 model option.

    Open sourceAPI / SDKCLILocal modelsOwn API key
    60 2.2M down 5.8% 121K 4d ago 2.2M downloads a week down 5.8% 30dGid 60 Free
  2. 21
    Browserbase · Browser · browser automation library

    Open-source SDK from Browserbase for building browser agents that mix natural-language actions (act, extract, observe, agent) with code, available in TypeScript, Python and Go and runnable locally or on Browserbase cloud browsers.

    Open sourceAPI / SDKOwn API key
    57 2.9M up 39.5% 26.1K 2d ago 2.9M downloads a week up 39.5% 30dGid 57 Free + $20/mo
  3. 139
    Skyvern AI · Browser · browser automation platform

    Open-source platform that automates browser-based workflows with LLMs and computer vision, available as a self-hosted AGPL project with Python and TypeScript SDKs and as the managed Skyvern Cloud, which adds proxies and anti-bot measures.

    Open sourceAPI / SDKCLIWeb appLocal modelsOwn API key
    34 8,843 up 15.5% 23.5K 5d ago 8,843 downloads a week up 15.5% 30dGid 34 $29/mo
  4. 157
    Amazon · Browser · browser automation service

    AWS service and Apache-2.0 Python SDK for building and running fleets of UI-automation agents that complete browser workflows from natural-language act() calls plus Python code and hand off to a human when needed. Workflows are developed in a web playground, IDE extension or CLI and deployed on AWS.

    Open sourceAPI / SDKCLIWeb appMCP
    30 116K up 24.8% 920 4d ago 116K downloads a week up 24.8% 30dGid 30 Usage-based
  5. —
    Clay · Sales · go-to-market data platform with AI research agents

    Go-to-market data platform for building and enriching prospect lists from many data providers, with Claygent and account agents that research companies and people on the web.

    Web app
    — — — — — Website rank #30.6K (Tranco top-sites list) Free + $167/mo
  6. —
    Perplexity AI · General · autonomous task agent

    Perplexity's general-purpose agent that takes a goal in natural language and deploys subagents across several frontier models to research, build apps, websites and reports, and work in connected apps, including Slack and Microsoft 365. A Personal Computer mode works on local files on Mac and Windows.

    Web appDesktop appMobile app
    — — — — — No public usage numbers —

“#” is the position among products of the same kind (agents, frameworks, tooling) by Gid Score, our 0–100 composite of adoption, community, attention and momentum. Use the filters and sorting above, or switch to the table view. Methodology · All AI agents.

Agent or classic scraper

A classic scraper is fast and cheap but breaks when a page changes. A browser agent reads the page the way a person does, so it copes with layout changes and multi-step flows (search, filter, paginate) at a higher cost per page. Many teams use agents to build or repair extraction steps and classic code to run them at volume.

What each agent does for this task

Only agents whose own documentation describes this use are listed. Checked Oct 6, 2026.

Agent What it does Source
Browser Use Open-source browser agent that returns extracted web data as structured output docs
Stagehand extract() pulls structured data from web pages against a schema docs
Skyvern Browser automation that extracts structured data from websites against a schema docs
Amazon Nova Act act_get() extracts information from web pages, with an optional response schema docs
Clay Claygent researches and scrapes websites to enrich rows in a table docs
Perplexity Computer Automates browser actions and extracts data from websites docs

Stay within the rules

  • Read the site's terms and robots.txt. Many sites forbid automated collection; an agent does not change that.
  • Prefer an official API where one exists.
  • Do not collect personal data you have no lawful basis to process, and rate-limit what you run.

More options by job: the best AI agents in every category.