litellm Alternatives

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

General-purpose agent in one static Go binary: console TUI, ACP server for editors, OpenAI-compatible API with embedded web UI, Telegram gateway, cron scheduler, swarm relay, remote mode. Rules, skills, subagents, hooks, MCP, background tasks, long-term memory, context compaction. Any OpenAI-compatible provider, distroless-ready.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Self-hosted LLM gateway that routes requests across AI providers (OpenAI, Anthropic, Gemini, Mistral, Ollama) using intelligent multi-policy scoring — including an LLM-native routing policy. Drop-in compatible: just swap the base URL. No database required, built-in cost tracking, budget enforcement and multi-tenant isolation.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Self-hosted AI gateway for Amazon Bedrock. OpenAI, Anthropic, Cohere and Ollama compatible APIs, plus Amazon Polly, Transcribe and Comprehend.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Open-source security gateway for LLM APIs — prompt injection detection, PII redaction, dangerous response sanitization, and audit logging. OpenAI/Claude compatible, MCP & Agent SKILL support. Drop-in proxy for AI coding agents (Cursor, Claude Code, Codex).

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

MCP Testing Tool — Like Postman, but for the Model Context Protocol.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Enterprise-ready MCP Gateway & Registry that centralizes AI development tools with secure OAuth authentication, dynamic tool discovery, and unified access for both autonomous AI agents and AI coding assistants. Transform scattered MCP server chaos into governed, auditable tool access with Keycloak/Entra integration.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF.

Same mcp server intent with mcp, llm_gateway overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Local OpenCode-backed LLM gateway for OpenAI, Anthropic, Gemini, and Responses API-compatible tools, with streaming and tool/function calling.

Same mcp server intent with mcp overlap.

Fit: Strong replacement candidate with overlapping indexed use cases.

Command a fleet of AI agents, on every deck, every desk, every wrist. Self-hosted multi-agent platform with memory, a coding pipeline, Flows, a team kiosk and voice devices. Any LLM, or 100% local with Ollama.