Promptfoo
Open-source solution for automated LLM prompt, agent, and RAG workflow testing, enabling red teaming, pentesting, and vulnerability scanning. Supports model comparisons across major AI providers with declarative configs, CI/CD, CLI, live reload, and web-based result sharing.
Cost / License
- Freemium
- Open Source (MIT)
Platforms
- Online
- Self-Hosted
Promptfoo News & Activities
Recent activities
Promptfoo information
Developed by
promptfoo Licensing
Open Source (MIT) and Freemium product.Pricing
free version with limited functionality.Written in
Alternatives
1 alternatives listedSupported Languages
- English
AlternativeTo Category
DevelopmentGitHub repository
- 22,594 Stars
- 2,004 Forks
- 357 Open Issues
- Updated
What is Promptfoo?
Promptfoo is an open-source tool for testing, evaluating, and securing LLM applications. It helps developers compare prompts, models, agents, and RAG workflows through repeatable automated evaluations rather than manual trial and error.
Use Promptfoo to run side-by-side tests across providers such as OpenAI, Anthropic, Azure, Amazon Bedrock, Ollama, and others; define test cases and assertions in declarative configuration files; inspect results in a web viewer or command line; and share findings with a team. It also supports AI red teaming and vulnerability scanning to identify issues such as unsafe behavior, prompt-injection risks, and compliance concerns before deployment.
Promptfoo is designed for developer workflows, with local execution, caching, live reload, and CI/CD integration so evaluation and security checks can run automatically during development and pull-request review. It is MIT-licensed and available as a CLI and library.