Beautiful Soup vs lxml
In 1,621 Python projects set up for AI coding agents, Beautiful Soup is a dependency in 135 (8.3%) and lxml in 80 (4.9%): Beautiful Soup is 1.7× as common. 58 projects list both, often a sign of a migration or of a monorepo with several apps.
Scraping, browsers and parsing PyPI Updated Nov 17, 2026
| Beautiful Soup | 8.3% | |
|---|---|---|
| lxml | 4.9% |
Share of 1,621 Python projects set up for AI coding agents that list each one as a dependency.
Side by side
| Beautiful Soup | lxml | |
|---|---|---|
| Agent projects that use it | 135 | 80 |
| Rank in scraping, browsers and parsing | #1 | #3 |
| In new repos with a scraping, browsers and parsing tool (last 90 days) | — | — |
| In older repos with one | 45.2% | 26.4% |
| Downloads, last 7 days | 73.9M | 98M |
| GitHub stars | — | 3,075 |
| Last push to its repo | — | Nov 15, 2026 |
| License | MIT | BSD-3-Clause |
| Funding | — | — |
| Projects that list both | 58 | |
Downloads include CI runs and installs as a dependency of other packages. Stars and pushes are from GitHub's API; downloads from PyPI.
By coding agent
| Projects set up for | Python repos | Beautiful Soup | lxml |
|---|---|---|---|
| Claude Code | 782 | 9.5% | 5.4% |
| GitHub Copilot | 805 | 7.2% | 4.6% |
| Cursor | 548 | 9.1% | 4.9% |
| Gemini CLI | 83 | 10.8% | 0% |
| OpenAI Codex | 78 | 10.3% | 0% |
| OpenCode | 53 | 9.4% | 0% |
| OpenClaw | 104 | 13.5% | 4.8% |
Share of the Python repos configured for each agent (its config file in the repo) that list the tool. A repo set up for two agents counts for both.
Beautiful Soup
Python HTML/XML parsing and screen-scraping library for web data extraction.
Most often used with it:
| Requests | 72.6% of its projects |
|---|---|
| python-dotenv | 68.1% of its projects |
| FastAPI | 51.1% of its projects |
| Uvicorn | 50.4% of its projects |
| Pydantic | 49.6% of its projects |
pip install beautifulsoup4
lxml
High-performance XML and HTML processing library combining libxml2/libxslt with Python.
Most often used with it:
| Beautiful Soup | 72.5% of its projects |
|---|---|
| python-dotenv | 60.0% of its projects |
| Requests | 60.0% of its projects |
| Pydantic | 48.8% of its projects |
| FastAPI | 42.5% of its projects |
pip install lxml
More matchups
How we count
Projects are public GitHub repos our weekly scan finds through coding-agent config files (CLAUDE.md, AGENTS.md, .cursor/mcp.json and others); a project uses a tool when its package.json, requirements.txt or pyproject.toml lists it. "Same job" was checked for every pair, so add-ons and companions are never paired. Shares by repo age compare repos created in the last 90 days with older ones in the same scan. More on the method.