operatorLM

OperatorLM is a lightweight local alternative to container-heavy proxies like LiteLLM. Packaged as a single ~11 MB Go binary with zero Docker dependencies (saving ~500 MB of RAM), it routes OpenAI-compatible requests across multiple LLM providers with automatic failover. It listens on 127.0.0.1:11434, the default Ollama port, and exposes an OpenAI-compatible API at /v1. An alias maps one model name to an ordered list of providers and accounts: if a target hits a 429 rate limit or fails, the request moves seamlessly to the next. Retries with backoff and jitter, a circuit breaker and a per-target RPM limiter are tunable from an embedded admin UI. API keys are stored securely in the OS keyring (Windows Credential Manager, macOS Keychain or Linux Secret Service), eliminating .env sprawl. Providers include OpenAI, OpenRouter, Groq, Gemini, Anthropic, Mistral and Bedrock. It can also run local GGUF models through llama-server (llama.cpp) and Whisper/Piper speech. Zero telemetry, single-user local development.