promptfoo
Tests, evaluates and red-teams LLM prompts and apps.
3. Used by agents' projects
Dependency in 9 public GitHub projects set up for AI coding agents that we scanned (0.1%).
4. Agent access
Agent skills for this library in the skills catalog; community skills are written by others, read them before you install.
From the package registry and GitHub, checked daily. Tap a verdict to see its source.How we checkedReport an error
What promptfoo does
Tests, evaluates and red-teams LLM prompts and apps.
promptfoo popularity
Gid Score measures public popularity and activity, not quality. How the Gid Score is calculated
How promptfoo is used
- promptfoo is a dependency in 9 of the 6,255 public GitHub projects set up for AI coding agents that we scanned, too few to describe it from them alone. Our scan also samples projects by dependency (Solana 3,064, Ethereum and EVM 1,779, 3D and games 3,123, agent frameworks and SDKs 13,665); counting those, it appears in 35 projects, and what it is used with below comes from all of them.
- The most used alternative in our scan is Evalite (2 projects).
- Projects that use it most often also use tsx (82.9%), Playwright (JavaScript) (77.1%), Vitest (80.0%).
By coding agent
Often used with
More about promptfoo
Funding
- GitHub Sponsors: https://github.com/typpo
- 3 GitHub sponsors
Agent Skills for it
- promptfoo-evaluation: Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconf
More in Prompts, structured output and evals
Alternatives
promptfoo This page
TypeScript eval runner for LLM apps, built on Vitest.
Get started
Install it.
npm install promptfooNo step-by-step start for promptfoo yet: we have not written its hand-checked card.