Skip to content
Agents tracked: 492 Releases (7d): 545 Agent status: 1 with issuesSince last limit reset: Claude 9d · Codex 7d For AI agents: Markdown Updated Nov 20, 2026

Evaluate

HuggingFace library for evaluating machine learning models with standardized metrics.

Prompts, structured output and evalsPYPI

GitHubWebsite

1. License

License from PyPI classifier.

github.com/huggingface/evaluate

From the package registry and GitHub, checked daily. Tap a verdict to see its source.How we checkedReport an error

What Evaluate does

HuggingFace library for evaluating machine learning models with standardized metrics.

Evaluate popularity

48 / 100
Gid Score, #2041 overall
1.1M
downloads a week, pypi
2,497
GitHub stars
4
agent projects use it
Evaluate: downloads per weekPyPI, evaluate 971,659
Evaluate: GitHub starshuggingface/evaluate 2,497

Gid Score measures public popularity and activity, not quality. How the Gid Score is calculated

How Evaluate is used

  • Evaluate is a dependency in 4 of the 6,254 public GitHub projects set up for AI coding agents that we scanned, too few to describe it from them alone. Our scan also samples projects by dependency (Solana 3,063, Ethereum and EVM 1,777, 3D and games 3,120, agent frameworks and SDKs 13,629); counting those, it appears in 63 projects, and what it is used with below comes from all of them.
  • The most used alternative in our scan is Ragas (3 projects).
  • Projects that use it most often also use Hugging Face Datasets (88.9%), NumPy (81.0%), pandas (81.0%).

Often used with

In 88.9% of its projects
In 81.0% of its projects
In 81.0% of its projects
In 79.4% of its projects
In 73.0% of its projects
In 68.3% of its projects
In 66.7% of its projects
In 66.7% of its projects

More about Evaluate

Agent Skills for it
  • evaluate: Use when the user wants proof that finished work meets its acceptance criteria, not just that tests pass: 'verify it is actually done', 'che
  • evaluate-presets: Evaluate and audit Ralph's built-in presets: run the evaluation scripts, check hat routing, review a preset's design. Use after changing a p
  • evaluate-rag: Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring r

See all

More in Prompts, structured output and evals

Alternatives

Evaluate This page

Apache-2.0Gid 481.1M/wk

Ragas

Evaluate RAG and LLM applications with automated metrics and scoring.

Apache-2.0Gid 52283K/wk

Compare Evaluate with another tool

Get started

Install it.

pip install evaluate

No step-by-step start for Evaluate yet: we have not written its hand-checked card.