Agent frameworks & infrastructure
LLM gateways & API proxies
4,018 agents · most starred first
LLM gateways & API proxies, page 39 of 41
Rate-limited, batching, failover-capable async client layer for LLM APIs
A small, provider-agnostic gateway for OpenAI/Anthropic-style language and vision models.
AIYou distribution of LiteLLM with HTTPX2 and OpenAI Python SDK 3+ support
Provider isolation for aqven: every model provider SDK import behind one factory.
Package name reserved — implementation coming soon
One-shot migration of inference from supported providers to Amazon Bedrock for Python repositories
Anthropic provider for BindAI
Groq provider for BindAI
Ollama provider for BindAI
A drop-in semantic cache for LLM APIs: match requests by meaning, serve repeats instantly, and route across OpenAI, Gemini, Groq, Claude, Ollama and Bedrock.
The production toolkit for LLM calls: caching, retries, fallback, cost tracking, security and structured output in one decorator.
国内大模型选择器/路由器:自然语言任务分类 × 评分矩阵推荐 × OpenAI 兼容网关路由
OpenAI-compatible /v1/chat/completions bridge backed by the DeepSeek Harness (dsh) Python SDK.
Zero-dependency OpenAI-compatible proxy that adapts DeepSeek Harness (dsh) to any OpenAI-compatible endpoint (Socrates/LiteLLM/sglang/vLLM)
Local proxy connecting coding agents to OpenAI-compatible AI providers
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.
Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.
Native LangChain integration for the ML Junction gateway
LangChain chat model for the NuRoute inference gateway
Declarative synchronization CLI for LiteLLM models, credentials, access groups and virtual keys.
A Textual TUI for managing LiteLLM proxy models
Serve Claude Max, ChatGPT Plus/Codex and Google Antigravity subscriptions as OpenAI-compatible models through a LiteLLM proxy
Official DeepSeek API package for llm-api-adapter
Kimi / Moonshot package for llm-api-adapter
Qwen Model Studio package for llm-api-adapter
Official Z.ai GLM API package for llm-api-adapter
llm plugin for APIMaster and other OpenAI-compatible gateways, including image and video generation
ChatGPT Codex subscription provider for llm-async
One transform spec, compiled to Python/Rust/TypeScript, translating LLM requests between provider formats.
Self-hosted AI gateway for Mac — one OpenAI-compatible LAN endpoint serving local MLX models with cloud API key fallback.
A local web UI for managing LLM API keys
A simple transparent MITM proxy for LLM APIs with a live Web UI
OpenAI-compatible LLM client with multi-provider fallback.
Additional files for the Hanzo AI proxy runtime. Kept separate so the main hanzo-llm package stays small.
Model adapters for llmgrid
Local multi-provider LLM proxy, model router and control plane for Claude Code, Codex, OpenCode, Gemini CLI and other AI coding agents: 60 providers, fallback routing, credential health, a native web-search tool proxy, and analytics with cost provenance
OpenAI-compatible Moderations API gateway backed by Ollama (Cloud or self-hosted)
An unofficial drop-in OpenAI Python SDK with a local SQLite response cache
OpenAI-compatible endpoint bridge for the Orditect ecosystem: governed LLM calls with audit, budget, and observability (bridge reference implementation)
Client for pankhllm, the LLM gateway that learns to skip the LLM.
The LLM gateway that learns to skip the LLM: trains its own tiny decision model on your agent's traffic and answers repeat decisions in 0.2 ms with no LLM call. OpenAI-compatible, one Rust binary. The server binary for pip users: pankhllm serve
Anthropic plugin for PolyLLM.
PolyLLM Python core: unified protocol, registry, and parameter policy.
DeepSeek plugin for PolyLLM.
OpenAI plugin for PolyLLM.
OpenAI-compatible endpoint plugin for PolyLLM.
A minimal, type-safe Python library for LLM chat completions with OpenAI, Azure OpenAI and Codex
Escalate a Pydantic AI agent to a stronger model when output validation keeps failing.
Universal OpenAI-compatible sidecar proxy with a live ragfaith faithfulness cascade
A reliability layer for LLM API calls: retries, rate limiting, validation, tool calling, fallbacks, and structured logging.
A unified async interface for multiple LLM providers (OpenAI, Ollama, OpenRouter) with event-based streaming, reasoning support, and multimodal inputs.
Client for the Zanii LLM API: OpenAI-compatible inference with a verifiable receipt for every call
APIQIK gives developers one OpenAI-compatible API for calling models such as OpenAI, Claude, and DeepSeek. It uses pay-as-you-go billing, automatic failover, sh
All-user-messages LLM provider plugin for DeepSeek Harness session titles
DeepSeek api-key provider authentication and discovery
DeepSeek account provider authentication and discovery
Local LLM gateway that lets TypeSafe's Jev model pick the tool for coding agents. Launchers for Codex, Claude Code, OpenCode, Kilo, Gemini and Devin, with a monitoring dashboard.
Fast-Path Gateway and Calibrated Decision Router for AI Agents (System 1 Decision Engine)
Framework-agnostic core for using Claude Code's OAuth subscription as an Anthropic provider: credential read/refresh, signed billing header, beta computation, and request transforms. No Pi, no LangChain — just the native Claude Code stack.
OpenAI-compatible server for Cogitator agents: Chat Completions, Responses and Assistants APIs
An OpenAI compat integration for Effect
Local OpenAI Responses, Chat Completions and tool-use workflows. Replies use configured scenarios or labeled deterministic stubs; no model runs.
The OpenAI-compatible chat-completions provider as a Stratus plugin: registers `openai-compatible`, selectable by a soul, credentialed per agent through the plugin seam
Manifest gateway management CLI (mnfst) — configure agents, providers, and routing from the terminal
DSH provider plugin for Codex access through a ChatGPT subscription.
OpenAI-compatible HTTP gateway for Agent Client Protocol coding agents
A generic pi provider extension for OpenAI-compatible, Anthropic-compatible, OpenAI Responses, and Ollama chat endpoints
Auth that enables users to sign in with their own OpenAI API key, so that you can build apps powered by users. Your app, their API usage.
Server-only SDK for the Lovable AI Gateway
AIT: proxy, planner, dan orkestrasi multi-agent untuk Claude Code, Codex CLI, Gemini CLI, opencode, dan Qwen Code (Windows x64)
Multi-provider, multi-agent free model aggregator and coding assistant
Official AgentMuxer SDK: search and call APIs with one key, one balance and a maximum cost on every call, from TypeScript or your agent framework
High-fidelity model calls for DeepSeek Harness: auto-adapts rejected optional parameters and keeps token usage accounting faithful
Lightweight production middleware for resilient LLM calls: retries, fallback routing, circuit-breaking, and rigid schema validation.
Failover and credential-routing layer for AI agent stacks: probe agents with real requests, classify failures with a zero-token gate, switch on collapse, fail back on recovery.
Provider-agnostic contracts for AI LLM Kernel.
Shared implementation for AI LLM Kernel providers.
Provider selection and dynamic loading for AI LLM Kernel.
Moves to another candidate after a retryable failure.
Anthropic provider for AI LLM Kernel.
Cohere provider for AI LLM Kernel.
Google GenAI provider for AI LLM Kernel.
Groq provider for AI LLM Kernel.
LM Studio provider for AI LLM Kernel.
Mistral provider for AI LLM Kernel.
Ollama provider for AI LLM Kernel.
OpenAI provider for AI LLM Kernel.
Chooses a client and model before the call.
Name reserved by Blugrid Pty Ltd for a private package. This placeholder does nothing.
AWS-native resilience layer for Amazon Bedrock: cross-region and cross-model failover with schema validation.
Per-user, per-feature and daily spend limits, plus loop detection, for OpenAI and Anthropic SDK calls.
Sign in with ChatGPT: CLI and Python API for a single stored OAuth connection
Sign in with ChatGPT (ChatGPT plan usage) for locally hosted apps
Check usage quota, rate limits and remaining sessions across LLM API providers (Z.ai GLM, Kimi/Moonshot).
Route Claude Code tasks to the cheapest sufficient model and effort