Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 36 of 41

hiperrouter

HiperRouter CLI - Start and manage HiperRouter AI Agent & Server

@metaharness/router

Cost-optimal model router for AI agent harnesses — route each query to the cheapest model that's good enough (k-NN over labelled embeddings). The productized DRACO Phase-2 finding.

umans-gate

LLM capture proxy with Anthropic cache_control TTL stamping, vision handoff, concurrency gating, rate limiting, and a live inspection dashboard

opencode-llm-proxy

Local AI gateway for OpenCode with tool/function calling — use any model via OpenAI, Anthropic, or Gemini API format

@nitra/llm-lib

Тонкий шар роботи з LLM (локальні omlx + хмарні провайдери) поверх pi: model tiers, one-shot, agentic-раннери, write-guard, trace, telemetry, prompt-budget

@rulvar/anthropic

Rulvar first-class provider adapter over @anthropic-ai/sdk.

@rulvar/bridge-ai-sdk

Rulvar bridge adapter wrapping any Vercel AI SDK LanguageModelV4 as a ProviderAdapter.

@punktechnologies/sdk

Gateway Agnostic SDK for Punk's hosted AI gateway: OpenAI, Anthropic, OpenRouter, Vercel AI SDK, LangChain, agent tracing, tool caching, and governance.

gosce-authenticated-mcp-jwt-agent

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

cursor-openai

Drop-in OpenAI-shaped client for Cursor models (LangChain / LangGraph optional). Pass apiKey + model.

@the-next-ai/ai-gateway

TypeScript Fastify AI protocol gateway for OpenAI, Anthropic, Gemini, MCP and agent workflows.

openai-oauth

Free OpenAI API access with your ChatGPT account.

@fudrouter/fsrouter

9Router v2 — Monorepo (frontend + backend)

grok-cli-to-openai-compatible

OpenAI-compatible API gateway for Grok CLI — with Admin panel, safe mode, and gctoac CLI

@pilotrun/provider-openai-compatible

OpenAI-compatible language model adapter for Pilot (Ollama, OpenAI, OpenRouter, and similar endpoints).

@bman654/clodex

Bridge Claude Code to OpenAI and OpenCode Go models — API keys or ChatGPT/Codex-plan OAuth, with prompt caching, auto-compaction, model switching, and a Claude Code binary patcher

@humanlayer/agentlayer-provider-openai-codex

OpenAI Codex provider for AgentLayer and the AI SDK. It talks to the ChatGPT Codex responses endpoint (`https://chatgpt.com/backend-api/codex/responses`) and supports ChatGPT OAuth/API-key auth through `@humanlayer/agentlayer-provider-auth`. Every factory

@rynfar/meridian

Local Anthropic API powered by your Claude Max subscription. One subscription, every agent.

@truss-harness/provider-openai-compatible

OpenAI-compatible streaming model provider for Truss.

@openai-oauth/core

Core auth and in-memory OpenAI-compatible transport for openai-oauth.

cursor-agent-bridge

Responses-compatible API bridge for Cursor Agent CLI

@muthuishere/routsi

OpenAI-compatible dynamic LLM router — a single Go binary, installed via npm as a prebuilt native binary

pi-openai-api-models-sync

Pi extension that syncs models from an OpenAI-compatible API and enriches them with pricing and capability metadata.

@holoscript/llm-provider

Unified LLM provider SDK for HoloScript - OpenAI, Anthropic, Gemini, OpenRouter, xAI, and local LLM adapters

opencode-models-discovery

OpenCode plugin for auto-discovery of OpenAI-compatible models with dynamic provider configuration

@useautumn/gateway

Autumn usage tracking adapters for AI SDKs and gateways (Vercel AI SDK, OpenRouter)

@cline/llms

Config-driven SDK for selecting, extending, and instantiating LLM providers and models

agentic-server

Standalone OpenAI-compatible LLM gateway — multi-provider proxy with pluggable inference metering

agensflow-langgraph

AgensFlow adapter for LangGraph — decorator-based routing across a declared model pool

agent-nonsense

A local zero-token agent status API with OpenAI and Anthropic compatible streaming

agent-wire-proxy

Protocol bridge for OpenAI Responses, Chat Completions, and Anthropic Messages clients.

agenticstack-cache

LLM response cache for AgenticStack: provider wrapper with in-memory LRU and SQLite backends

agentwarden-ai

A local OpenAI-compatible proxy that reduces agent context waste and proves behavior held.

brink-litellm-plugin

Reserved namespace placeholder. The real brink-litellm-plugin is not yet public — see README.

cascade-llm

LLM Pipeline Query Optimizer — automatic model routing, cost optimization, and quality-aware escalation.

chatgpt-oauth

Experimental subject-scoped ChatGPT OAuth and subscription transport

dv-llmclient

An LLM client with cache and type checking

edenai-haystack

Use Eden AI with Haystack to build AI applications with 500+ AI models through one unified, OpenAI-compatible API.

factory-droid-openai

Unofficial OpenAI-compatible HTTP bridge for Factory Droid

fllme

A functionnal oriented Python CLI for LLM calling with multi provider and multi model

gosce-authenticated-llm-agent-anthropic

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

gosce-authenticated-llm-agent-bcrypt

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

gosce-authenticated-mcp-agent-langchain-core

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

gosce-authenticated-mcp-agent-openai

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

gosce-authenticated-mcp-agent-opentelemetry-api

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

image-api-openai

A proxy service converts ModelScope and SiliconFlow image APIs to OpenAI-compatible format

langchain-claude-cli

Drop-in ChatAnthropic replacement backed by the Claude Code CLI — use your Claude Pro/Max subscription, no API key needed

lite-llm-doctor

LiteLLM / OpenAI-compatible gateway doctor CLI (ping, tokens, context bench)

litellm-free-image-providers

Custom LiteLLM providers for Pollinations.ai and AI Horde, two free image-generation APIs

litellm-plugin-openrouter-pareto

Pareto-optimal OpenRouter provider chooser for LiteLLM

litellm-routing-testbed

Local proof harness for LiteLLM context-window routing

liter-llm-hermes-plugin

Universal LLM API client for 174 providers — chat, streaming, tools, embeddings, search, OCR, plus an OpenAI-compatible proxy and an MCP server.

llama-index-llms-iflytek

llama-index llms iflytek spark integration

llm-api-resilience

A resilient multi-provider LLM layer that automatically fails over to alternative models and providers when requests fail.

llm-budget-guard

Cross-vendor LLM quota / budget self-check CLI. Stable 0/1/2/3 exit-code contract lets any AI agent gate its work before burning through budget. Supports MiniMax Token Plan, OpenRouter credits, self-hosted LiteLLM proxy spend, and self-logged JSONL (any LLM vendor).

llm-cascade

Free-first LLM routing for Python: local Ollama -> OpenRouter free models -> paid Anthropic API, with automatic fallback and a savings ledger.

llm-catalogue

Model names, pricing, and free-tier metadata for OpenAI, Anthropic, and Google Gemini.

llm-chat-completions-server

LLM plugin to serve an OpenAI Chat Completions API endpoint

llm-circuit-breaker

A tiny, readable circuit breaker for LLM calls: hard budget caps that fail over to a free local model instead of raising or overspending.

llm-cost-governor

Composable pre-call and post-call hooks for LLM API calls: pricing, budgets, cost caps, rate limits, event log, observability.

llm-meta-ai

LLM plugin for the Meta AI API

llm-perplexity-agent

Perplexity's Agent API as a model for Simon Willison's llm CLI.

llm-pycascade

Resilient cascading LLM inference across multiple providers with failover, circuit breaking, and retry cooldowns

llm-rate-governor

Client-side rate limiting for the Claude API. Never sees your API key.

llm-routewise

Cost-aware, latency-optimized routing for multiple LLM API providers

llm-unorouter

LLM plugin for models hosted by UnoRouter

llm-verse-avneesh

Async LLM router across AWS Bedrock, Google Gemini, and Groq providers

llm-zai

LLM plugin for Z.AI / GLM models with Coding Plan support

llm-zai-glm

LLM plugin for Z.AI / GLM models with Coding Plan support

llm-zenmux

LLM plugin for ZenMux.ai - OpenAI Chat Completions + Anthropic Messages API

llmguard-py

A lightweight Python decorator for caching, rate limiting, and retrying LLM API calls.

llmrouterx

Intelligent routing, load balancing, and failover for LLM providers

llmswitch-client

A production-ready routing, fallback, and rate-limiting client for OpenAI-compatible LLM endpoints.

lmux-bedrock-shared

Shared AWS Bedrock internals (SigV4 signing, event-stream decoding, Anthropic pricing) for lmux

login-with-chatgpt

Use the OpenAI Python SDK with your own ChatGPT subscription

nbchatgpt

ChatGPT-5.4 Nano 中转服务 SDK - 每天免费 1.0K tokens,国内开箱即用

neutral-llm-gateway

Neutral LLM gateway: typed contracts, provider adapters and explicit retry, fallback, usage and cost accounting.

openai-compatible-errors

Normalize OpenAI-compatible API failures, plan conservative retries, redact logs, and inspect SSE streams

polyllmkit

Streaming, multi-provider LLM bridge + markdown rendering toolkit for terminal apps.

proxyllm-client

Client helper for ProxyLLM, the OpenAI-compatible LLM gateway: one-line OpenAI client wiring plus the autonomous email-OTP account signup flow.

quillm

A personal AI model-routing stack that learns from your usage

routesmith-llm

Adaptive LLM execution engine - intelligent routing, cascading, caching, and budget management

seamoney-hetu-llm-control-plane

A small example package

sefia-litellm

LiteLLM adapter for Sefia.

shopee-litellm

A small example package

sotto-llm

Structured thinking as a pluggable layer for any OpenAI-compatible API

specllm

Makes any LLM behave like a regular REST API. Define the contract, specllm enforces it.

swapllm

Swap between LLM providers (Groq, OpenAI, Anthropic) through one interface, with automatic fallback and optional schema validation.

waldur-site-agent-envoy-ai-gateway

Envoy AI Gateway (API keys + usage reporting) plugin for Waldur Site Agent

warpllm

A warp-speed, robust AI gateway

watchllm-orchestrator

WatchLLM runtime orchestration and model routing.

wyoming-openai-gateway

A gateway that exposes local Wyoming services via OpenAI-compatible API endpoints

@gosce/authenticated-llm-agent

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

@gosce/authenticated-llm-agent-anthropic

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

@gosce/authenticated-llm-agent-bcrypt

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

@gosce/authenticated-multi-llm-agent

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

@gosce/authenticated-llm-agent-google-auth-oauthlib

Google-OAuth-gated LLM gateway: verify a Google ID token, then run a Gemini (Vertex AI) completion for the verified caller.

llm-relay

Loopback bidirectional Anthropic/OpenAI API proxy with tool-call validation and multi-provider routing.

@gosce/authenticated-mcp-agent-langchain-core

JWT-gated LLM gateway: authenticate (bcrypt/JWT), then run a LangChain-on-Vertex Gemini completion. Unauthenticated calls are rejected.

@noir-ai/model

Noir model — optional bounded single-shot LLM layer (anthropic / openai / openai-compatible); agent loops are impossible by design.

More in Agent frameworks & infrastructure