Overmind

Overmind is an open-source model training platform for AI teams. It takes the OpenTelemetry traces your agent already emits and turns them into a smaller specialised model you own, trained and benchmarked with your own evaluators.

Features:

• Observability: OTLP trace ingest with no collector, span explorer, sessions and live scoring on incoming spans.
• Datasets: built from production traces, file uploads, or imports from Langfuse, LangSmith, Braintrust and Galileo, with provenance back to every source trace.
• Eval: five evaluator kinds (LLM judge, deterministic, trajectory, statistical, agentic) shared by batch experiments and production scoring.
• Optimiser: rewrites prompts, tool definitions and agent logic as scored git diffs.
• Training: LoRA or full fine-tuning of 38 open-weight models including Qwen, Llama and Gemma, benchmarked with your evaluators.
• Inference: OpenAI-compatible serving of trained and frontier models, with downloadable weights.

The platform is AGPL-3.0 with an MIT-licensed Python SDK and CLI. Self-host it with Docker Compose or use the hosted version, which has a free plan.