Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 40 of 41

claude-wrap

A model router for Claude Code.

gemini-model-runner

A lightweight Python wrapper for Google's Gemini API

gemini-router

Quota-aware Gemini router with persistent usage ledger, native API proxy, TTS and auto-updating CLI

gosce-encrypted-oauth-llm-agent

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

gosce-oauth-langchain-openai-agent

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

groq-gemini-bridge

Chatbot library that calls Groq and Gemini with multi-key automatic failover.

headroom-ollama

Headroom proxy extension: compress Ollama traffic under its own /ollama route

hermes-claude-bridge

Hermes provider plugin for the pi-claude-cli Claude worker

hermes-nanogpt-provider

Hermes Agent plugin for NanoGPT as model provider.

kirollm

Python and LangChain client for Kiro's streaming LLM API

langchain-acedatacloud

Ace Data Cloud chat model and shared LangChain service runtime

langchain-aistupidlevel

LangChain chat model for the AI Stupid Level Smart Router: each request goes to the model that scores best for the task, with your own provider keys.

langchain-bitdeer-ai

An integration package connecting Bitdeer AI and LangChain

langchain-model-router

In-process model routing for LangChain: a chat model that picks one of several candidate chat models per request.

langchain-otari

LangChain chat models served through an Otari gateway

langchain-runcrate

LangChain integration for Runcrate: open models behind one OpenAI-compatible API

langchain-topxai

An integration package connecting TopxAI and LangChain

litellm-brownout

Tiered team budget enforcement and graceful downgrade for a LiteLLM gateway

litellm-core

Library to easily interface with LLM API providers

litellm-preflight

Flags LiteLLM proxy settings that cost money or break at scale, like background health checks multiplied across every process

llama-index-llms-acedatacloud

Native Ace Data Cloud LLM integration for LlamaIndex

llama-index-llms-blockrun

LlamaIndex LLM for BlockRun: one OpenAI-compatible gateway, paid per call in USDC over x402 on Base or Solana. No API key.

llama-index-llms-runcrate

LlamaIndex LLM integration for Runcrate: open models behind one OpenAI-compatible API

llm-catalog-ai-sdk

AI SDK for Python adapter for llm-catalog: build native `ai` models from a catalog config, routed through your gateway.

llm-flexai

LLM plugin for models hosted by FlexAI

llm-mesh

LLM clients with a shared model catalog

llm-openai-decisions

LLM plugin for the OpenAI Decisions API

llm-proxy-cli

A lightweight CLI tool for delegating LLM tasks to expert models across multiple providers.

llm-public-ai

LLM models from publicai.co

llm-route-audit

Prove it before you route: test cheaper LLMs on your own traffic and switch only where quality holds.

llm-router-lite

Send each prompt to the cheapest model that can handle it, and fall back when one fails

llm-runcrate

LLM plugin for Runcrate: open models behind one OpenAI-compatible API

llm-transport

Run a prompt through the claude CLI or the Anthropic Messages API (direct or via a LiteLLM gateway), with one switch and one result shape.

llm-trustedrouter

LLM plugin for models available through TrustedRouter

llmfuse

Reliability for LLM calls: retries with backoff, circuit breakers, rate limiting and multi-provider failover.

llmroutes

Benchmark-driven router for OpenAI-compatible LLM endpoints

llmstream

Lightweight multi-provider LLM streaming parser with sentence buffering for Voice AI, tag guarding, and streaming tool-call assembly. Zero runtime dependencies.

llmswitch-sdk

Unified Python client for chat, embeddings and the Responses API across OpenAI, Azure OpenAI, Azure AI Foundry, Anthropic and Vertex AI, with rate limits, retries, failover, and optional agents with tools.

llmwarden

Call any model with any key, fetched safely from wherever you keep secrets, and see exactly where your money goes

mistralai-capabilities-deployment-apps

The Mistral Apps gateway seam: the caller token the gateway puts on each request, the middleware that catches it, and the host wiring that makes it the credential a caller's Mistral API calls spend.

my-llm-py

Profile-driven async LLM adapter for OpenAI and Anthropic.

nl2data-openai

OpenAI structured-output provider for the nl2data-core model provider boundary.

ollama-decision

Python SDK for Ollama's Decision API. Convert text into structured, typed responses without prose generation using an elegant operator-based DSL.

otari-haystack

An integration between the Otari LLM gateway and Haystack

plyllm

One interface to many LLM providers and models: generation, structured output, embeddings, costs, retries and rate limits.

polyllm-gemini

Google Gemini plugin for PolyLLM.

pura-llm-sdk

Official Pura LLM OAuth and delegated inference SDK

pydantic-ai-model-selection

Per-request model selection, a feature for pydantic-ai-app-factory

pydantic-ai-otari

Pydantic AI models served through an Otari gateway

pyflowred-node-ai-openai

OpenAI for PyFlowRED's AI nodes - and anything that speaks the OpenAI API.

quantum-llm-client

A lightweight wrapper for querying an EC2-hosted LLM via OpenAI API

runcrate-haystack

Haystack integration for Runcrate: open models behind one OpenAI-compatible API

shad0wllm

Learns the fixed-choice decisions your LLM makes and answers the ones it is certified on in microseconds, from a small table on your CPU. Zero-code proxy for the OpenAI Decisions API, System One and chat models.

simllms

Unified client for multiple LLM providers

theagentrouter

LiteLLM's API with Agent Router underneath: the SDK and the local gateway.

theagentrouter-core

LiteLLM's API with Agent Router underneath: the SDK alone, for use with a remote gateway. pip install theagentrouter adds the local gateway.

theagentrouter-local-gateway

Agent Router's aigw program, for theagentrouter's local gateway. Installed with pip install theagentrouter.

thinwrap-llm

Lightweight, SDK-free, zero-dependency Python wrapper for chat completions (with streaming) and embeddings across 18 LLM providers behind one normalized surface.

thriftllm

LLM cost routing with correct per-request cost: output-length forecasting, cache-aware tiered pricing, and $/quality ranking.

tripwire-litellm

Per-client budget hard stop and cost logging for LiteLLM proxies, by Tripwire

xyberos-openai-compatible

A ModelProvider for OpenAI-compatible chat and embedding endpoints, proved against the shipped conformance suite

quiet-core-bot

Quiet Core bot - a self-hosted AI agent gateway

claude-nim-proxy

Thin Claude Code to NVIDIA NIM translator

claude-roulette

Use several Claude subscriptions from one Claude Code setup via a local proxy

llm-openrouter-decisions

LLM plugin for the decision models of the OpenRouter Decisions API

llmadapterevs

A unified Python client for OpenAI-compatible and multi-provider LLM APIs.

pydantic-ai-cloudflare-ai

Cloudflare Workers AI and AI Gateway models for Pydantic AI

alimbo

An AI agent gateway for tool orchestration, automation, and chat integrations.

@hydraharness/harness-llm

Provider-neutral LLM service interface for the Hydra harness

@agentxin-ai/plugin-moonshot

`@agentxin-ai/plugin-moonshot` connects [Moonshot AI (Kimi)](https://platform.moonshot.cn/) models to the [AgentXinAI](https://github.com/agentxin-ai/agentxin) platform. The plugin integrates Moonshot's OpenAI-compatible API so AgentXinAI agents can lever

@henryqw/pi-multi-codex

Use multiple ChatGPT Codex OAuth accounts in Pi.

tinfoil

Tinfoil secure OpenAI client wrapper

opencode-plugboard

Use the ChatGPT, Claude, Copilot and OpenCode Go logins in your OpenCode server from any app: an OpenCode V2 plugin serving OpenAI, Anthropic and Gemini compatible APIs.

@zenning/provider
@lighton-ai/sdk

LightOn TypeScript SDK

mixedbread-ai-provider

Mixedbread AI Provider for running Mixedbread AI models with Vercel AI SDK

@entrogic-net/providers

Model providers for BanglaClaw: OpenAI-compatible (OpenAI, Ollama, vLLM) and Anthropic

langchain-heabsy

LangChain chat model for Heabsy, an OpenAI-compatible inference API for open models

llm-parry

Zero-config rate limiting, caching, and cost control for LLM calls.

@camstack/addon-ai

AI addon for CamStack — the `llm` collection provider (cloud, LAN, and camstack-managed local llama.cpp profiles) plus the per-node `llm-runtime` managed executor.

@vouchington/ai

Provider-neutral model-call primitives: usage, pricing, JSON-output adapters, and structured-decision codecs.

@magpie-community/opencode-cursor-auth

Cursor subscriptions (Pro, Pro+, Ultra, Teams) in OpenCode and magpie, through the API cursor-agent talks to

claude-relay-server

Claude Code API server: OpenAI- and Anthropic-compatible endpoints for Claude Code, using your Claude subscription

dark-litellm

Library to easily interface with LLM API providers

langchain-obscurify

LangChain chat model for Obscurify.ai, a private AI API (OpenAI-compatible)

llm-failover-chain

Multi-provider LLM failover with rate-limit handling and automatic retry.

llm-obscurify

LLM plugin for Obscurify.ai, a private AI API with 200+ models

taskiq-llm-queue

An LLM request client with its own durable queue: taskiq + Amazon SQS, adaptive limits, retry and fallback.

@crvouga/mockingbird-service-openai

Local scripted OpenAI emulator: chat/tool loops, SSE, embeddings, models, files and staged uploads without billed inference.

@tardie/ai-bedrock

AWS Bedrock Converse provider for Effect AI.

agent-harness-orchestrator

Route sub-tasks to the right model instead of the biggest one: a from-scratch orchestrator with a blackboard, strict JSON contracts and an auditable execution trace.

anyray-agent

Keeps an agent answering when its Anyray gateway is unreachable: an httpx2 transport for the OpenAI, Anthropic and LangChain SDKs that resends a call straight to the provider only when the gateway never handled it.

django-llm-cache

Production-grade, drop-in caching utility for LLM API prompts and responses in Django, cutting token costs and latency.

guac-llm-bridge

Bridge remote LLM APIs and SSH over an Apache Guacamole terminal

langchain-revana

LangChain integration for Revana: Claude, Gemini, DeepSeek and other chat models through one OpenAI-compatible API key.

llm-ai-api

A thin, robust HTTP wrapper for the OpenAI, OpenRouter, Anthropic (Claude), and Google APIs (only dependency: httpx)

llmgift

Gift someone a spending-capped, expiring LLM API key. The tool is published on npm: npx llmgift

rboto-bedrock

rboto: async-only, typed Python client for Amazon Bedrock, powered by Rust. Alpha preview.

rboto-bedrock-runtime

rboto: async-only, typed Python client for Amazon Bedrock Runtime, powered by Rust. Alpha preview.

saturnus-llm

Lightweight LLM connectivity and protocol compatibility tools.

More in Agent frameworks & infrastructure