Apps tagged with 'LLM Evaluation'

Public platform for evaluating large language models using anonymous pairwise comparisons, crowd-sourced voting, real-time result updates, and global performance tracking.

Open-source LLM-ops platform combining prompt management, an AI gateway, tracing, a tool catalog, and evaluation in one self-hostable platform.

Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more.

Helicone is the open-source LLM observability platform for developers to monitor, debug, and improve production-ready applications.

The reliability platform for AI agents - trace, evaluate, guard, and control the cost of every LLM call and agent handoff in production.

Netra is the reliability platform for AI agents to observe, evaluate, simulate, and continuously improve every decision your agents make, so you can ship with confidence and catch regressions before your users do.

Feel how fast a local LLM runs before you buy the hardware.

Test AI models yourself, privately, with a standardized benchmark, and get both technical scores AND practical recommendations.

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

Forecall is for people who build MCP servers and APIs that AI agents call. It checks the text an agent reads before it picks a tool, and measures what models actually do with it.

Lisapet.ai is the next-level AI product development platform that empowers teams to prototype, test, and ship robust AI features 10x faster.

Open-source tool for automated LLM prompt, agent, and RAG evaluation, supporting red teaming, multi-model comparisons, CI/CD, CLI, and vulnerability scanning.

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

The platform for building on AI.

A free, open-source app to discover, run, and use AI models on your own computer, with hardware-aware recommendations, easy setup, and advanced controls when needed.

Open-source platform that turns your LLM agent’s production traces into a specialised model you own, trained, benchmarked and served.

Unified platform for routing LLM traffic, observability, evals, prompt management, tracing, and spend controls, reducing dashboard switching time.

AI agent evaluation and benchmarking platform — simulate, evaluate, and continuously improve agents before they reach production.

Open-source LLM observability and evaluation: grade every AI agent trace, cluster the failures, and replay them as regression tests that block the pull request.

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.