Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 28 of 41

llm-fcio
llm-fiber

A thin, fast, observable Python client for LLMs

llm-fireworks

Access fireworks.ai models via API

llm-forge-playground

Small CLI playground for Qwen via an OpenAI-compatible LLM Forge gateway.

llm-forwarder

A configurable forwarder for OpenAI-compatible LLM requests

llm-gateway

This is LLM Gateway package

llm-gcp-vertex

LLM plugin for Google Cloud Vertex AI models including Gemini and Model Garden

llm-gemini

LLM plugin to access Google's Gemini family of models

llm-gemini-code-assist

LLM plugin to access Google's Gemini via Code Assist API with OAuth

llm-gigachat
llm-github-copilot

GitHub Copilot plugin for LLM CLI tool

llm-github-models

LLM plugin to access GitHub Models API

llm-greenpt

A plugin to add support for the GreenPT AI inference service to llm

llm-grok

LLM plugin providing access to Grok models using the xAI API

llm-groq
llm-handler

A unified handler for LLM API calls across OpenAI, Anthropic, and Google

llm-horde

Plugin for LLM adding models hosted on AI Horde

llm-http-api

HTTP API for LLM with OpenAI compatibility

llm-hub-sdk

This Python SDK that provides convenient clients for interacting with the LLM-HUB – a centralized backend service

llm-huggingface

LLM plugin providing access to the huggingface inference API

llm-huggingface-plugin

LLM plugin to access Hugging Face models

llm-hyperbee

Access hyperbee.ai models via API

llm-hyperbolic

LLM access to Hyperbolic's API

llm-iaedu

LLM plugin for iaedu.pt API — no adapter required

llm-inception

Run prompts against LLMs hosted by Inception Labs with diffusing animation

llm-inference-cloud

LLM plugin for models hosted by Inference

llm-inference-router

Pluggable LLM inference routing SDK — bring any model from any provider

llm-interface

A flexible interface for working with various LLM providers

llm-io-normalizer

A lightweight Model I/O normalization layer for OpenAI-compatible LLM calls.

llm-jina

LLM plugin for Jina AI

llm-jlm
llm-json

A JSON library wrapper that gracefully handles markdown-wrapped JSON from LLMs.

llm-json-adapter
llm-json-fix

使用 LLM 修正 json 文件

llm-json-fixer

A simple utility to fix malformed JSON from LLM outputs

llm-json-guard

Production-safe JSON repair and schema validation for LLM outputs

llm-json-repair

Robust JSON parsing for LLM outputs with automatic repair and field extraction

llm-json-streaming

A unified interface for streaming structured JSON from OpenAI, Anthropic, and Google Gemini.

llm-json-utils

Fast, minimal JSON repair implemented in Rust (PyO3), focusing on truncation and trailing comma fixes.

llm-keypool

Free-tier LLM API key pool with rotation, cooldown, and LangChain drop-in

llm-kit-pro

A unified toolkit for working with multiple LLM providers

llm-lambda

LLM access to LAMBDA's API

llm-lambda-labs

Run prompts against LLMs hosted by https://lambdalabs.com/

llm-limiters

A thread-safe and async rate limiter for Gemini and OpenAI models.

llm-list

A Python package to list and monitor available LLM models from various providers

llm-litellm

LLM plugin for LiteLLM proxy server

llm-litellm-proxy

LLM plugin for LiteLLM OpenAI-compatible proxies

llm-mimetypes

LLM MIME types

llm-minimax

LLM plugin for accessing MiniMax models via their API

llm-mistral

LLM plugin providing access to Mistral models using the Mistral API

llm-model-alias

Standalone model alias resolver

llm-model-filter

Filter llm models list by starring favorites

llm-model-gateway

OpenAI-compatible API server for simonw's llm cli

llm-model-router

Intelligent multi-LLM request router for cost optimization

llm-models

A CLI tool to list available LLM models from various providers

llm-moonshot

Run prompts against LLMs hosted by Moonshot

llm-multiprovider

LLM API for multiple model providers

llm-neuralwatt

A plugin to add support for the Neuralwatt AI inference service to llm

llm-nexus

A unified interface for multiple language model providers

llm-nosrun

LLM plugin for models hosted by NOS Cloud

llm-nous

Access Nous Research models via API

llm-onesdk

OneSDK is a Python library that provides a unified interface for interacting with various Large Language Model (LLM) providers.

llm-only

Conditionally serve content to LLM crawlers in Python web frameworks.

llm-openai-codex

Access OpenAI models via a Codex subscription

llm-openai-plugin

LLM plugin for OpenAI

llm-openai-via-codex

Access OpenAI models via a Codex subscription

llm-openrouter

LLM plugin for models hosted by OpenRouter

llm-palm

Plugin for LLM adding support for Google's PaLM 2 model

llm-parametrizer

A library to parametrize multiple API calls for LLMs

llm-predictive-router

A package to route chat requests between LLMs based on prompt classification

llm-provider

A unified interface for multiple LLM providers

llm-provider-factory

A unified interface for multiple LLM providers with image generation, speech-to-text, and function calling (OpenAI, Anthropic, Gemini, VertexAI, Ollama + DALL-E, Replicate, Whisper, Google Speech, Tools)

llm-providers-list

A unified source of truth for LLM providers, models, pricing, and capabilities.

llm-proxier

A lightweight LLM proxy with logging and admin dashboard

llm-proxy-ondemand

A FastAPI proxy server for vLLM w/wo SLURM support and automatic process management

llm-proxy-server

LLM Proxy Server is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

llm-proxypy

llm proxy server

llm-questioncache

A cacheing CLI for quickly asking questions of LLMs

llm-quiver

The auxiliary tool is used to invoke the online LLM service API, supporting local caching, prompt rendering, and configuration management.

llm-rate-guard

Cloud-agnostic rate limit mitigation for LLM APIs

llm-registry

A centralized registry for discovering and managing LLM model capabilities. Track model features, costs, and limitations across providers like OpenAI and Anthropic. Supports both verified model definitions and user-managed entries with local storage.

llm-reka

Access Reka models via the Reka API

llm-relay

Unified LLM usage management — API proxy, session diagnostics, multi-CLI orchestration.

llm-replicate

LLM plugin for models hosted on Replicate

llm-req

llm req client

llm-requesty

LLM plugin for models hosted by requesty

llm-response
llm-retry-py

Exponential backoff with full jitter for LLM API calls. Sync + async. Built-in retryable-code presets for Anthropic, OpenAI, Bedrock, Gemini. Zero runtime deps.

llm-rosetta

LLM-Rosetta: Unified LLM API adapter — bidirectional conversion between OpenAI, Anthropic, and Google formats via intermediate representation (IR).

llm-rotate

Unified LLM client with key rotation, health-aware failover, and multi-provider orchestration.

llm-rotator

Fault-tolerant LLM provider rotation with circuit breaker, quotas, and model-first routing

llm-route

Intelligent LLM routing with weighted least-outstanding selection, 429 cooldown, and request class isolation

llm-router

Quick LLM routing using Embeddings

llm-router-ledger

Route text and embedding calls across LLM providers, keeping a JSONL ledger of every request and response for offline cost reconciliation.

llm-rovodev

LLM plugin exposing the RovoDev model for the llm CLI

llm-samba

LLM plugin providing access to the free Samba models using the samba API

llm-sambanova

LLM plugin for models hosted by SambaNova

llm-sdk-provider-gemini

Gemini provider plugin for llm-sdk.

llm-selector

A Python library for selecting the best LLM model based on user input using any LLM via LiteLLM

llm-smart-route

輕量級 LLM 模型追蹤與自動替換 Decorator

More in Agent frameworks & infrastructure