Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 29 of 41

llm-smart-router

Intelligent tool routing for LLMs with 95.8% reduction in context size

llm-sniffer

LLM Sniffer - OpenAI-compatible reverse proxy with request/response inspector

llm-space

A Python wrapper for managing OpenAI API and other LLM models.

llm-std-lib

Python library for standardization, optimization and monitoring of LLM API calls

llm-straico
llm-structured-output-Detect-repairs

一个简单的HTTP客户端库,用于与结构化输出API交互

llm-switch

LLM Switch Module - 一個用於切換不同 LLM 提供商的模組

llm-switchboard

A provider-agnostic Python interface for routing generation calls across LiteLLM, OpenAI, Ollama, and Watsonx

llm-taxi

Call LLM as easily as calling a taxi.

llm-throttle-guard

Token-bucket rate limiter for LLM API calls with per-model limits

llm-together

llm together plugin

llm-togetherai

LLM plugin for models hosted by Together AI

llm-token-optimizer

Token cost control and auto-optimization for LLM apps — compress prompts, estimate costs, enforce budgets, route to cheap models, and cut LLM spend by up to 60%

llm-token-surgeon

Cut your LLM API bill by 30-70% with zero accuracy loss

llm-unified-interface

适配国内主流大语言模型的统一访问接口

llm-unified-sdk

Python SDK for the Unified LLM Chat Backend API.

llm-uv-tool

A plugin for LLM that enables proper plugin management when installed via uv tool.

llm-venice

LLM plugin to access models available via the Venice API

llm-verse

A unified Python client to call any LLM models from multiple providers

llm-vertex

Plugin for LLM adding support for Google Cloud Vertex AI

llm-vertex-fork

Plugin for LLM adding support for Google Cloud Vertex AI

llm-watsonx

LLM plugin for IBM watsonx models

llm-wrap
llm-wrapper-test1

A comprehensive Python wrapper for Large Language Models with database integration and usage tracking

llm-wrapper-testing

A comprehensive Python wrapper for Large Language Models with database integration and usage tracking

llm-wrapper-yl

A flexible wrapper for LLM providers with caching, async support, and cost tracking

llm-xai

LLM plugin for models hosted by OpenRouter

llm-zhipu

ChatGLM

llm2openai

Create a Python package.

llm7-validator

Validator for LLM7 chat completion requests with Pydantic and model ID constraints.

llm7token

Utilities for validating and reporting LLM7 API tokens.

llmUnify

LlmUnify is a library that abstracts connections to major LLM model providers, simplifying their invocation and interoperability.

llm_handler_validator

LLMHandler is a unified Python package that provides a single, consistent interface for interacting with multiple LLM providers, offering both structured (typed) and unstructured responses.

llm_providers

Common interface for language models.

llm_store

LLM Gateway Service

llmabstractions

A Python library abstracting the way you interact with LLMs.

llmadapter

LLMs Adapter Package

llmai

Unified Python client for OpenAI, Azure OpenAI, Vertex AI, Anthropic, Gemini, DeepSeek, OpenRouter, Cerebras, Fireworks, Together AI, LM Studio, Bedrock, LiteLLM, and ChatGPT.

llmaid

A zero‑dependency wrapper that turns any OpenAI‑compatible endpoint into a hackable one‑liner.

llmakits

A powerful Python toolkit for simplifying LLM integration and management with multi-model scheduling, fault tolerance, and load balancing support

llmanger

Add your description here

llmao-py

Lightweight LLM API Orchestrator - A fast, multi-provider LLM client with rate limiting and key rotation

llmapi

Do you want to talk directly to the LLMs? Try llmapi.

llmapi-cli

Do you want to talk directly to the LLMs? Try llmapi.

llmbridge-py

Unified Python interface for OpenAI, Anthropic, Google, and Ollama LLMs with optional model registry and usage tracking - deprecated

llmbroker
llmcachex

Intelligent caching layer for LLM calls with semantic understanding, async deduplication, and cost control

llmcachex-ai

Drop-in semantic + exact caching layer for LLM applications

llmcake

LLM API calls made piece of cake🍰

llmcall

A lite abstraction layer for LLM calls

llmcaller

Python SDK for LLMCaller - A unified interface for LLM providers

llmchat

大语言模型仿openai接口实现

llmconf

LLMConf unifies the loading and generation configurations of various LLM backends (e.g. `openai`, `transformers`, `vllm`), simplifying parameter control for experiments and testing.

llmcostcut

A framework for selectively invoking LLMs and distilling repeated workloads into smaller models

llmctl

A professional CLI tool for interacting with OpenAI and Anthropic LLMs with cost tracking

llmcycle

Cycle through multiple LLM providers with smart fallback, load balancing, and unified API with real-time FastAPI dashboard, resilient streaming failover, Pydantic structured output, prompt caching, token budgets, and local/cloud key rotation.

llmdog

轻量级 LLM 命令行工具与 Python 库,支持兼容 OpenAI Chat Completions 格式的任意 API 端点。

llme-cli

A command-line interface for language models

llmem

Read through cache for OpenAI's API.

llmfleet

Fleet-level batch dispatcher for LLM APIs. Pool requests across coroutines, route to provider Batch APIs, save 50% on cost without rewriting your agent loops.

llmfusion

Unified interface for multiple LLM providers

llmgate

Lightweight, provider-agnostic LLM calling library — OpenAI, Gemini, Anthropic, Groq, Mistral, Cohere, Azure, Bedrock, Ollama

llmgateway-sdk

LLMGateway Python SDK Client

llmgt

Plug-and-play LLM connector via YAML config

llmhosts

Your Personal AI Cloud -- intelligent proxy, router, and cache for LLMs

llmhub
llmhub-cli

LLMHub is a lightweight management platform designed to streamline working with LLMs.

llmhub-gateway

LLMHub - Open-source multi-provider LLM gateway

llmhub-runtime

A lightweight runtime for LLMHub that delegates to any-llm

llmhub.ai

LLMHub - Open-source multi-provider LLM gateway

llming-models

LLM execution engine — multi-provider streaming, MCP tools, and budget management

llmkit-py

Unified LLM API client (placeholder)

llmkit-python

Production-grade LLM client for Python - 100+ providers, 11,000+ models. Rust-powered.

llmkit-rs

Unified LLM API client (placeholder)

llmkit-sdk

AI API gateway SDK with cost tracking, budget enforcement, and multi-provider routing for LLM applications

llmlib

Portability lib for OpenAI, Hugging Face, and FastChat local model APIs

llmlib-amq

A simple library for making requests to LLMs

llmlibrary

A Python library for interacting with Large Language Models (LLMs).

llmling-provider-llm

LLMling provider based on the package 'llm'

llmlite

A library helps to chat with all kinds of LLMs consistently.

llmmaster

A unified interface for interacting with multiple LLMs and generative AIs.

llmonkey

A library to talk to different language models

llmopenai

Create a Python package.

llmplus

A package for using llm models with api

llmpool

Large Language Models' pool management library

llmpop

The Python library that lets you spin up any LLM with a single function. Route between local and remote LLMs with a unified interface.

llmprompt-nexus

A unified framework for interacting with various LLM providers through a consistent interface with batch processing, templating, and rate limiting.

llmproxy
llmproxy-withlog

OpenAI-compatible LLM reverse proxy with real-time conversation analytics

llmquery

A package for querying various LLM providers

llmring

Unified Python interface for OpenAI, Anthropic, Google, and Ollama LLMs

llmring-server

LLMRing core server: alias sync, receipts, and usage logging (key-scoped, no users)

llmrouter-lib

A unified framework for LLM routing and evaluation.

llms-Arunsingh

A unified interface for multiple LLM providers

llms-py

A lightweight CLI tool and OpenAI-compatible server for querying multiple Large Language Model (LLM) providers

llms-wrapper

Package for simple usage of diverse LLMs and configuring them using human readable config files

llmsays

One-line LLM queries with smart routing and provider failover

llmservice

A lightweight, production-ready service layer for modular, rate-aware LLM integrations

llmshim

Multi-provider LLM gateway — one interface, every provider

llmspec

LLM unified interface

More in Agent frameworks & infrastructure