Prefactor

Prefactor is the Agent Evaluation Runtime for AI agents.

We give engineering teams a single place to observe, evaluate and compare how their agents are performing across development and production.

Prefactor is focused on four core areas:

• Agent Observability Capture every agent run, trace, span and conversation, with visibility into latency, token usage, cost, models, tools and agent behaviour.

• Continuous Evaluation Evaluate live agent traffic for quality, task success, risk and other signals as agents run, rather than relying only on offline datasets or sampled traces.

• Versions & Environments Track agent versions across development, staging and production, and compare how quality, latency, cost and behaviour change between releases, environments and deployments.

• Production Feedback Loops Detect failures and regressions, drill into the underlying runs, and use real production behaviour to create new evaluation scenarios and improve the next version of the agent.

Teams can also segment and compare performance by agent, version, environment, trace, team and other metadata, making it easier to understand exactly where behaviour changed and what caused it.

Prefactor integrates through Python and TypeScript SDKs, API and CLI, giving teams an agent-first runtime for understanding not just what their agents are doing, but whether they are getting better or worse over time.