Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 39 of 41

aiollm-gateway

Rate-limited, batching, failover-capable async client layer for LLM APIs

aiyallm

A small, provider-agnostic gateway for OpenAI/Anthropic-style language and vision models.

aiyou-litellm

AIYou distribution of LiteLLM with HTTPX2 and OpenAI Python SDK 3+ support

aqven-llm

Provider isolation for aqven: every model provider SDK import behind one factory.

axm-llm

Package name reserved — implementation coming soon

bedrock-migrate

One-shot migration of inference from supported providers to Amazon Bedrock for Python repositories

bindai-provider-anthropic

Anthropic provider for BindAI

bindai-provider-groq

Groq provider for BindAI

bindai-provider-ollama

Ollama provider for BindAI

cachellm-proxy

A drop-in semantic cache for LLM APIs: match requests by meaning, serve repeats instantly, and route across OpenAI, Gemini, Groq, Claude, Ollama and Bedrock.

callm-toolkit

The production toolkit for LLM calls: caching, retries, fallback, cost tracking, security and structured output in one decorator.

cn-llm-router

国内大模型选择器/路由器:自然语言任务分类 × 评分矩阵推荐 × OpenAI 兼容网关路由

dsh-openai

OpenAI-compatible /v1/chat/completions bridge backed by the DeepSeek Harness (dsh) Python SDK.

dsh-openai-shim

Zero-dependency OpenAI-compatible proxy that adapts DeepSeek Harness (dsh) to any OpenAI-compatible endpoint (Socrates/LiteLLM/sglang/vLLM)

free-claude-code

Local proxy connecting coding agents to OpenAI-compatible AI providers

gosce-encrypted-langchain-session-agent

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

gosce-google-oauth-langchain-agent

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

gosce-google-sign-in-langchain-agent

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

gosce-langchain-login-and-token-agent

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

gosce-oauth-multi-llm-orchestrator

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

gosce-openai-login-and-token-agent

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

gosce-traced-openai-langchain-agent

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

langchain-mljunction

Native LangChain integration for the ML Junction gateway

langchain-nuroute

LangChain chat model for the NuRoute inference gateway

litellm-auto-synch-manager

Declarative synchronization CLI for LiteLLM models, credentials, access groups and virtual keys.

litellm-controller

A Textual TUI for managing LiteLLM proxy models

litellm-mysubs

Serve Claude Max, ChatGPT Plus/Codex and Google Antigravity subscriptions as OpenAI-compatible models through a LiteLLM proxy

llm-api-adapter-deepseek

Official DeepSeek API package for llm-api-adapter

llm-api-adapter-kimi

Kimi / Moonshot package for llm-api-adapter

llm-api-adapter-qwen

Qwen Model Studio package for llm-api-adapter

llm-api-adapter-zai

Official Z.ai GLM API package for llm-api-adapter

llm-apimaster

llm plugin for APIMaster and other OpenAI-compatible gateways, including image and video generation

llm-async-codex

ChatGPT Codex subscription provider for llm-async

llm-babel

One transform spec, compiled to Python/Rust/TypeScript, translating LLM requests between provider formats.

llm-ferry

Self-hosted AI gateway for Mac — one OpenAI-compatible LAN endpoint serving local MLX models with cloud API key fallback.

llm-keys-ui

A local web UI for managing LLM API keys

llm-mitm-proxy

A simple transparent MITM proxy for LLM APIs with a live Web UI

llm-provider-sdk

OpenAI-compatible LLM client with multi-provider fallback.

llm-proxy-extras

Additional files for the Hanzo AI proxy runtime. Kept separate so the main hanzo-llm package stays small.

llmgrid-network

Model adapters for llmgrid

my-claude-code

Local multi-provider LLM proxy, model router and control plane for Claude Code, Codex, OpenCode, Gemini CLI and other AI coding agents: 60 providers, fallback routing, credential health, a native web-search tool proxy, and analytics with cost provenance

ollama-moderation-gateway

OpenAI-compatible Moderations API gateway backed by Ollama (Cloud or self-hosted)

openai-sqlite-cache

An unofficial drop-in OpenAI Python SDK with a local SQLite response cache

orditect-bridge-openai

OpenAI-compatible endpoint bridge for the Orditect ecosystem: governed LLM calls with audit, budget, and observability (bridge reference implementation)

pankhllm

Client for pankhllm, the LLM gateway that learns to skip the LLM.

pankhllm-server

The LLM gateway that learns to skip the LLM: trains its own tiny decision model on your agent's traffic and answers repeat decisions in 0.2 ms with no LLM call. OpenAI-compatible, one Rust binary. The server binary for pip users: pankhllm serve

polyllm-anthropic

Anthropic plugin for PolyLLM.

polyllm-core

PolyLLM Python core: unified protocol, registry, and parameter policy.

polyllm-deepseek

DeepSeek plugin for PolyLLM.

polyllm-openai

OpenAI plugin for PolyLLM.

polyllm-openai-compatible

OpenAI-compatible endpoint plugin for PolyLLM.

pycallm

A minimal, type-safe Python library for LLM chat completions with OpenAI, Azure OpenAI and Codex

pydantic-ai-escalation

Escalate a Pydantic AI agent to a stronger model when output validation keeps failing.

ragfaith-proxy

Universal OpenAI-compatible sidecar proxy with a live ragfaith faithfulness cascade

steadfast-llm

A reliability layer for LLM API calls: retries, rate limiting, validation, tool calling, fallbacks, and structured logging.

unified-llm-provider

A unified async interface for multiple LLM providers (OpenAI, Ollama, OpenRouter) with event-based streaming, reasoning support, and multimodal inputs.

zanii-llm-client

Client for the Zanii LLM API: OpenAI-compatible inference with a verifiable receipt for every call

APIQIK

APIQIK gives developers one OpenAI-compatible API for calling models such as OpenAI, Claude, and DeepSeek. It uses pay-as-you-go billing, automatic failover, sh

@deepseek-ai/dsh-session-title-all-prompts-llm

All-user-messages LLM provider plugin for DeepSeek Harness session titles

@deepseek-ai/dsh-llm-deepseek-api-key

DeepSeek api-key provider authentication and discovery

@deepseek-ai/dsh-llm-deepseek-account

DeepSeek account provider authentication and discovery

jev-gateway

Local LLM gateway that lets TypeSafe's Jev model pick the tool for coding agents. Launchers for Codex, Claude Code, OpenCode, Kilo, Gemini and Devin, with a monitoring dashboard.

@cubicmaldo/vaelis

Fast-Path Gateway and Calibrated Decision Router for AI Agents (System 1 Decision Engine)

@cgaravitoq/claude-code-core

Framework-agnostic core for using Claude Code's OAuth subscription as an Anthropic provider: credential read/refresh, signed billing header, beta computation, and request transforms. No Pi, no LangChain — just the native Claude Code stack.

@cogitator-ai/openai-compat

OpenAI-compatible server for Cogitator agents: Chat Completions, Responses and Assistants APIs

@effect/ai-openai-compat

An OpenAI compat integration for Effect

@volter/twin-openai

Local OpenAI Responses, Chat Completions and tool-use workflows. Replies use configured scenarios or labeled deterministic stubs; no model runs.

@stratusagent/provider-openai

The OpenAI-compatible chat-completions provider as a Stratus plugin: registers `openai-compatible`, selectable by a soul, credentialed per agent through the plugin seam

mnfst-gateway-cli

Manifest gateway management CLI (mnfst) — configure agents, providers, and routing from the terminal

@eddyskywalker/dsh-chatgpt-subscription

DSH provider plugin for Codex access through a ChatGPT subscription.

@alint-js/model-adapter-acp

OpenAI-compatible HTTP gateway for Agent Client Protocol coding agents

@indexyz/pi-custom-provider

A generic pi provider extension for OpenAI-compatible, Anthropic-compatible, OpenAI Responses, and Ollama chat endpoints

@genesis-xyz/ai

Auth that enables users to sign in with their own OpenAI API key, so that you can build apps powered by users. Your app, their API usage.

@lovable.dev/ai-gateway

Server-only SDK for the Lovable AI Gateway

aithread

AIT: proxy, planner, dan orkestrasi multi-agent untuk Claude Code, Codex CLI, Gemini CLI, opencode, dan Qwen Code (Windows x64)

flappycode

Multi-provider, multi-agent free model aggregator and coding assistant

@agentmuxer/sdk

Official AgentMuxer SDK: search and call APIs with one key, one balance and a maximum cost on every call, from TypeScript or your agent framework

dsh-llm-fidelity

High-fidelity model calls for DeepSeek Harness: auto-adapts rejected optional parameters and keeps token usage accounting faithful

agent-fallback-sentinel

Lightweight production middleware for resilient LLM calls: retries, fallback routing, circuit-breaking, and rigid schema validation.

agent-junction

Failover and credential-routing layer for AI agent stacks: probe agents with real requests, classify failures with a zero-token gate, switch on collapse, fail back on recovery.

ai-llm-kernel-contracts

Provider-agnostic contracts for AI LLM Kernel.

ai-llm-kernel-core

Shared implementation for AI LLM Kernel providers.

ai-llm-kernel-factory

Provider selection and dynamic loading for AI LLM Kernel.

ai-llm-kernel-fallback

Moves to another candidate after a retryable failure.

ai-llm-kernel-provider-anthropic

Anthropic provider for AI LLM Kernel.

ai-llm-kernel-provider-cohere

Cohere provider for AI LLM Kernel.

ai-llm-kernel-provider-google-genai

Google GenAI provider for AI LLM Kernel.

ai-llm-kernel-provider-groq

Groq provider for AI LLM Kernel.

ai-llm-kernel-provider-lmstudio

LM Studio provider for AI LLM Kernel.

ai-llm-kernel-provider-mistral

Mistral provider for AI LLM Kernel.

ai-llm-kernel-provider-ollama

Ollama provider for AI LLM Kernel.

ai-llm-kernel-provider-openai

OpenAI provider for AI LLM Kernel.

ai-llm-kernel-router

Chooses a client and model before the call.

app-claude-router

Name reserved by Blugrid Pty Ltd for a private package. This placeholder does nothing.

bedrock-fallback-engine

AWS-native resilience layer for Amazon Bedrock: cross-region and cross-model failover with schema validation.

budget-guard-llm

Per-user, per-feature and daily spend limits, plus loop detection, for OpenAI and Anthropic SDK calls.

chatgpt-auth

Sign in with ChatGPT: CLI and Python API for a single stored OAuth connection

chatgpt-plan-auth

Sign in with ChatGPT (ChatGPT plan usage) for locally hosted apps

check-llm-quota

Check usage quota, rate limits and remaining sessions across LLM API providers (Z.ai GLM, Kimi/Moonshot).

claude-code-router

Route Claude Code tasks to the cheapest sufficient model and effort

More in Agent frameworks & infrastructure