One gateway.
Every LLM provider.
Your infrastructure.
llmBreakr is a self-hosted AI gateway that puts OpenAI, Anthropic, and Gemini behind one OpenAI-compatible endpoint — with virtual keys, budgets, rate limits, and full audit logs. No traffic ever leaves your infra.
One endpoint in front of
Stop wiring every provider into every service
Point one client at your gateway. llmBreakr resolves the model, enforces limits, and normalizes the response — your application code doesn't change per provider.
import OpenAI from "openai";
import Anthropic from "@anthropic-ai/sdk";
import { GoogleGenerativeAI } from "@google/generative-ai";
const openai = new OpenAI({ apiKey: OPENAI_KEY });
const anthropic = new Anthropic({ apiKey: ANTHROPIC_KEY });
const gemini = new GoogleGenerativeAI(GEMINI_KEY);
// three SDKs, three auth models,
// three response shapes to normalize,
// raw keys hardcoded per serviceimport OpenAI from "openai";
const gateway = new OpenAI({
baseURL: "https://gateway.internal/v1",
apiKey: process.env.LLMBREAKR_VIRTUAL_KEY,
});
// one client, one shape, any provider —
// swap "model" and nothing else changes.
// budgets, limits, and audit logs
// are enforced by the gateway, not your code.Everything a production LLM surface needs
Built at the infrastructure layer, once — instead of re-implemented in every service that calls a model.
One unified endpoint
A single OpenAI-compatible /v1/chat/completions API, streaming included, in front of every configured provider.
Virtual API keys
Scoped, project-level credentials replace raw provider keys — with optional approval workflows before they go live.
Rate limits & budgets
Per-project request-rate ceilings and spend caps, enforced in Redis before a request ever reaches a provider.
Full audit trail
Every chat request and every administrative action is logged — know exactly who called what, and who changed what.
Role-based access
An RBAC system scopes what each admin account can configure, approve, or revoke inside the dashboard.
Management dashboard
A Next.js UI for projects, keys, providers, and monitoring — bundled in the same deployable as the gateway.
Automatic fallback
Configure alternate models per project — if the primary provider fails, the request escalates through your fallback chain instead of erroring out.
Every request earns its way to a provider
A short, fast pipeline inside a single Express app — control plane and data plane, split by responsibility and auth model.
Virtual key auth
Reject invalid or revoked keys immediately.
Model resolution
Check the project is allowed to call this model.
Limit enforcement
Rate limit + budget check, backed by Redis.
Provider adapter
Translate to OpenAI / Anthropic / Gemini shape.
Log & stream back
Audit trail written, response streamed to client.
Any check fails → request rejected before a provider is ever called, before any cost is incurred.
Live in under five minutes
Pull the published image, point it at your MySQL and Redis, and you have a working gateway with an admin dashboard on port 3000.
$git clone https://github.com/yashb007/llmBreakr-ai-gateway.git$cd llmBreakr-ai-gateway && cp server/.env.example .env$docker compose up -dOwn the gateway, own the data
Most AI gateways ask you to route production traffic through their platform. llmBreakr is the gateway itself — deployed on infrastructure you already control.
Swipe to see the full comparison →
Deploy your own gateway in the next five minutes.
MIT licensed, self-hosted, and running on infrastructure you already trust. Clone it, star it, or send a PR.