OBEY API Gateway

OBEY API Gateway is an open-source (MIT), self-hosted AI gateway that gives you a single OpenAI-compatible endpoint in front of all your LLM providers. Just change your OPENAI_API_BASE to point at the gateway with no application code changes and gain automatic failover, intelligent routing, and cost control across providers.

It ships as a single Rust binary with no runtime dependencies: download and run on Windows (installer or portable zip with a system-tray app), deploy with Docker, or one-click deploy to Railway.

Key features:

Drop-in OpenAI replacement — full /v1/* compatibility (chat, completions, embeddings, images, audio, Assistants API, and a native /v1/responses front door used by Codex CLI). Multi-provider routing — OpenAI, Ollama, AWS Bedrock, Groq, Together AI, NVIDIA NIM, vLLM, and LM Studio. Automatic failover — circuit breakers plus retry with exponential backoff across providers, including mid-stream failover. Smart rate-limit handling — instantly skips providers returning 429, honors Retry-After / X-RateLimit-Reset headers, and supports weekly-quota providers via per-provider cooldowns. Priority & cost-aware routing — configure model groups with priority, cost, and latency-based selection, plus complexity-aware tier selection (Fast / Balanced / Powerful). Streaming reliability — true SSE pass-through, early synthetic events for sub-500ms TTFB, keep-alive, graceful in-stream error frames, and truncation retry. Response caching — built-in in-memory exact-match cache plus an optional semantic (Qdrant) tier. Guardrail pipelines — pre-call and post-call policy enforcement with PII redaction, regex scanning, Presidio, OpenAI Moderation, Lakera, and custom HTTP providers. Virtual key management — issue per-caller keys with independent USD/token budgets, rate limits, model-access restrictions, and expiry, without sharing real provider keys. Encrypted key storage — provider keys encrypted at rest with a machine-local master key; OpenAI OAuth login supported. Token & tool-definition compression — multi-engine strategies to cut prompt and tool-schema token waste. Admin panel & dashboard — embedded web UIs for configuration, live metrics, in-flight request tracking, and log viewing, with hot config reload. Observability — Prometheus /metrics endpoint and SQLite-based structured request logging. Compared to hosted or Python-based gateways, OBEY runs as one native binary with the whole control plane (config, metrics, logs, keys) built into an embedded web UI, making it easy to self-host with zero external dependencies.