Arch

Arch handles the pesky low-level work in building AI agents like clarifying vague user inputs, routing prompts to the right agents, calling tools for simple tasks, and unifying access to large language models (LLMs) - all without locking you into a framework. Move faster by focusing on the high-level logic of your agents.

AI demos are easy to build. But past the thrill of a quick hack, you are left building, maintaining and scaling low-level plumbing code for agents that slows down AI innovation. For example:

• You want to build specialized agents, but get stuck building routing and handoff code.
• You want use new LLMs, but struggle to quickly and safely add LLMs without writing integration code.
• You're bogged down with prompt engineering work to clarify user intent and validate inputs.
• You're wasting cycles choosing and integrating code for observability instead of it happening transparently.

With Arch, you can move faster by focusing on higher-level objectives in a language and framework agnostic way. Arch was built by the contributors of Envoy Proxy with the belief that:

Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems to improve speed and accuracy for common agentic scenarios – all outside core application logic.*

Core Features:

• Routing to Agents. Engineered with purpose-built LLMs for fast (<100ms) agent routing and hand-off scenarios
• Routing to LLMs: Unify access and routing to any LLM, including dynamic routing via preference policies.
• Guardrails: Centrally configure and prevent harmful outcomes and ensure safe user interactions
• Tools Use: For common agentic scenarios let Arch instantly clarify and convert prompts to tools/API calls
• Observability: W3C compatible request tracing and LLM metrics that instantly plugin with popular tools
• Built on Envoy: Arch runs alongside app servers as a containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.