Git.Top Guide

Open Source LLM Gateways

Model routing, provider abstraction, OpenAI-compatible proxying, fallback, policy, and cost-control projects.

Guide

What To Inspect

Compare LLM gateways by provider support, OpenAI compatibility, routing controls, fallback behavior, policy hooks, deployment target, and observability integrations.

Guide

Best Use

Use this page when choosing infrastructure between AI applications and model providers, especially when teams need routing, budget control, provider failover, or a unified API surface.

Data Source

d1 / d1_query

1226 loaded projects. Generated at 2026-11-11T00:12:48.616Z.

Project

Hal0ai/hal0

63

Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI + systemd.

Llm GatewayLocalCloud
Quality31
Agent63

Project

VkRainB/ccMesh

63

Claude Code / Codex local AI proxy gateway desktop app (Tauri). Anthropic Messages <-> OpenAI-compatible protocol transform, model mapping, endpoint rotation/failover. 本地桌面代理网关:协议转换、模型映射、多端点轮换/熔断,对接官方 API 或中转站,不改客户端。Windows/macOS/Linux.

Llm GatewayLibrary OnlyLocalCloud
Quality24
Agent63

Project

sxueck/llm-gateway

61

Lightweight distributed LLM gateway w/ web UI for model mgmt & routing. Supports vibe programming, prompt opt., & optimized OpenAI API/Anthropic calls

Llm GatewayDockerLocalCloud
Quality22
Agent61

Project

izzoa/polyrouter

58

A self-hostable LLM router / gateway: OpenAI- and Anthropic-compatible, explicit-first routing with fallbacks, spend limits, and metadata-only cost tracking. No markup, no third-party proxy — your keys, your box.

Llm GatewayDockerLocalCloud
Quality21
Agent58

OpenAPI specification for the OpenAI API

Llm GatewayLocalCloud
Quality21
Agent69

High-performance proxy router for LLM APIs with automatic load balancing, rate limiting, and fail2ban protection

Llm GatewayDockerLocalCloud
Quality18
Agent60

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti

Llm GatewayLocalCloud
Quality11
Agent53

Project

BuilderIO/ai-shell

64

A CLI that converts natural language to shell commands.

Llm GatewayLibrary OnlyLocalCloud
Quality9
Agent64

Project

Nutlope/llamacoder

60

Open source Claude Artifacts – built with Llama 3.1 405B

Llm GatewayLibrary OnlyLocalCloud
Quality6
Agent60

OpenAI-compatible gateway aggregating 8 free LLM providers with automatic failover, circuit breakers, and smart routing. 32+ models, zero cost.

Llm GatewayDockerCloudflareServerless
Quality7
Agent56

Antigravity Proxy is a high-performance gateway that exposes Google's internal Gemini and Claude APIs through an OpenAI-compatible interface

Llm GatewayDockerLocalCloud
Quality6
Agent52