eliasbiondo / antirion

# Antirion
**The fastest open-source AI gateway.**
One Rust binary, with the admin console built in, between your apps and every model provider.
Performance ·
Getting started ·
Deploy ·
Features ·
Architecture ·
Docs
> [!TIP]
> **Faster than Bifrost and LiteLLM.** On the same dedicated machine, Antirion adds a fraction of
> Bifrost's overhead with a fraction of its CPU and memory, and keeps serving at rates where LiteLLM
> fails. See the benchmark.
## Why Antirion
- **The fastest gateway we have measured.** Sub-millisecond overhead at thousands of requests per
 second on 4 vCPUs, and several times the throughput of the next fastest gateway on the same
 machine (Performance). Rust, no garbage collector, no interpreter, one data
 thread per CPU.
- **One binary, console included.** The gateway, the Admin API and the web console ship as a
 single Rust binary (or a distroless image). No sidecars, no separate UI deployment, no runtime
 to install.
- **Every major provider behind one API.** OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock,
 Google Vertex AI, Gemini, Mistral, Cohere, Groq, Together, Fireworks, DeepSeek, xAI, Cerebras,
 DeepInfra, Perplexity, and any OpenAI- or Anthropic-compatible server such as vLLM. Clients
 speak OpenAI, Anthropic, Gemini or Bedrock; Antirion translates between them.
- **Routing that keeps you up.** Fallback chains, weighted splits, least cost, least latency,
 least busy, tiers, content classifiers, retries, hedging and circuit breakers, decided per
 request and recorded in the request log.
- **Spend under control.** Rate limits and budgets in dollars or tokens on the organization,
 team, project and key, with prices per Model, a cost ledger and alerts before a budget runs
 out.
- **Safety at the edge.** Guardrails for PII, secrets, keywords, regexes, JSON schemas, tool
 permissions, topics, moderation, model-based judges and partner services that catch prompt
 injection, on requests, responses and streams, with redaction that keeps the format.
- **Ready for production.** Exact and semantic caching, MCP and agent gateways, SSO with OIDC,
 SAML and SCIM, an audit log with change requests, and a standalone node or a cluster on
 PostgreSQL and Valkey.
## Performance
Antirion, Bifrost 2.2.4 and LiteLLM 1.103.1 ran on the same dedicated AWS c7i machines, interleaved
repetition by repetition, each with the same minimal setup: one key checked on every request, no
request logging, no cache. Every figure is the median of the repetitions.
**Simple chat at 5,000 requests per second, gateway on one c7i.xlarge (4 vCPUs)**
| | **Antirion** | Bifrost 2.2.4 | LiteLLM 1.103.1 |
| --------------- | ------------: | ------------: | --------------: |
| Overhead p50 | **0.53 ms** | 19.7 ms | — |
| Overhead p99 | **0.95 ms** | 95.2 ms | — |
| Errors | **0%** | 0% | 100% |
| CPU per request | **247 µs** | 717 µs | — |
| Peak memory | **215 MiB** | 1,389 MiB | 1,851 MiB |
| Idle memory | **41 MiB** | 498 MiB | 1,653 MiB |
**More scenarios, sam