Agent frameworks & infrastructure
LLM gateways & API proxies
4,018 agents · most starred first
LLM gateways & API proxies, page 29 of 41
Intelligent tool routing for LLMs with 95.8% reduction in context size
LLM Sniffer - OpenAI-compatible reverse proxy with request/response inspector
A Python wrapper for managing OpenAI API and other LLM models.
Python library for standardization, optimization and monitoring of LLM API calls
一个简单的HTTP客户端库,用于与结构化输出API交互
LLM Switch Module - 一個用於切換不同 LLM 提供商的模組
A provider-agnostic Python interface for routing generation calls across LiteLLM, OpenAI, Ollama, and Watsonx
Call LLM as easily as calling a taxi.
Token-bucket rate limiter for LLM API calls with per-model limits
llm together plugin
LLM plugin for models hosted by Together AI
Token cost control and auto-optimization for LLM apps — compress prompts, estimate costs, enforce budgets, route to cheap models, and cut LLM spend by up to 60%
Cut your LLM API bill by 30-70% with zero accuracy loss
适配国内主流大语言模型的统一访问接口
Python SDK for the Unified LLM Chat Backend API.
A plugin for LLM that enables proper plugin management when installed via uv tool.
LLM plugin to access models available via the Venice API
A unified Python client to call any LLM models from multiple providers
Plugin for LLM adding support for Google Cloud Vertex AI
Plugin for LLM adding support for Google Cloud Vertex AI
LLM plugin for IBM watsonx models
A comprehensive Python wrapper for Large Language Models with database integration and usage tracking
A comprehensive Python wrapper for Large Language Models with database integration and usage tracking
A flexible wrapper for LLM providers with caching, async support, and cost tracking
LLM plugin for models hosted by OpenRouter
ChatGLM
Create a Python package.
Validator for LLM7 chat completion requests with Pydantic and model ID constraints.
Utilities for validating and reporting LLM7 API tokens.
LlmUnify is a library that abstracts connections to major LLM model providers, simplifying their invocation and interoperability.
LLMHandler is a unified Python package that provides a single, consistent interface for interacting with multiple LLM providers, offering both structured (typed) and unstructured responses.
Common interface for language models.
LLM Gateway Service
A Python library abstracting the way you interact with LLMs.
LLMs Adapter Package
Unified Python client for OpenAI, Azure OpenAI, Vertex AI, Anthropic, Gemini, DeepSeek, OpenRouter, Cerebras, Fireworks, Together AI, LM Studio, Bedrock, LiteLLM, and ChatGPT.
A zero‑dependency wrapper that turns any OpenAI‑compatible endpoint into a hackable one‑liner.
A powerful Python toolkit for simplifying LLM integration and management with multi-model scheduling, fault tolerance, and load balancing support
Add your description here
Lightweight LLM API Orchestrator - A fast, multi-provider LLM client with rate limiting and key rotation
Do you want to talk directly to the LLMs? Try llmapi.
Do you want to talk directly to the LLMs? Try llmapi.
Unified Python interface for OpenAI, Anthropic, Google, and Ollama LLMs with optional model registry and usage tracking - deprecated
Intelligent caching layer for LLM calls with semantic understanding, async deduplication, and cost control
Drop-in semantic + exact caching layer for LLM applications
LLM API calls made piece of cake🍰
A lite abstraction layer for LLM calls
Python SDK for LLMCaller - A unified interface for LLM providers
大语言模型仿openai接口实现
LLMConf unifies the loading and generation configurations of various LLM backends (e.g. `openai`, `transformers`, `vllm`), simplifying parameter control for experiments and testing.
A framework for selectively invoking LLMs and distilling repeated workloads into smaller models
A professional CLI tool for interacting with OpenAI and Anthropic LLMs with cost tracking
Cycle through multiple LLM providers with smart fallback, load balancing, and unified API with real-time FastAPI dashboard, resilient streaming failover, Pydantic structured output, prompt caching, token budgets, and local/cloud key rotation.
轻量级 LLM 命令行工具与 Python 库,支持兼容 OpenAI Chat Completions 格式的任意 API 端点。
A command-line interface for language models
Read through cache for OpenAI's API.
Fleet-level batch dispatcher for LLM APIs. Pool requests across coroutines, route to provider Batch APIs, save 50% on cost without rewriting your agent loops.
Unified interface for multiple LLM providers
Lightweight, provider-agnostic LLM calling library — OpenAI, Gemini, Anthropic, Groq, Mistral, Cohere, Azure, Bedrock, Ollama
LLMGateway Python SDK Client
Plug-and-play LLM connector via YAML config
Your Personal AI Cloud -- intelligent proxy, router, and cache for LLMs
LLMHub is a lightweight management platform designed to streamline working with LLMs.
LLMHub - Open-source multi-provider LLM gateway
A lightweight runtime for LLMHub that delegates to any-llm
LLMHub - Open-source multi-provider LLM gateway
LLM execution engine — multi-provider streaming, MCP tools, and budget management
Unified LLM API client (placeholder)
Production-grade LLM client for Python - 100+ providers, 11,000+ models. Rust-powered.
Unified LLM API client (placeholder)
AI API gateway SDK with cost tracking, budget enforcement, and multi-provider routing for LLM applications
Portability lib for OpenAI, Hugging Face, and FastChat local model APIs
A simple library for making requests to LLMs
A Python library for interacting with Large Language Models (LLMs).
LLMling provider based on the package 'llm'
A library helps to chat with all kinds of LLMs consistently.
A unified interface for interacting with multiple LLMs and generative AIs.
A library to talk to different language models
Create a Python package.
A package for using llm models with api
Large Language Models' pool management library
The Python library that lets you spin up any LLM with a single function. Route between local and remote LLMs with a unified interface.
A unified framework for interacting with various LLM providers through a consistent interface with batch processing, templating, and rate limiting.
OpenAI-compatible LLM reverse proxy with real-time conversation analytics
A package for querying various LLM providers
Unified Python interface for OpenAI, Anthropic, Google, and Ollama LLMs
LLMRing core server: alias sync, receipts, and usage logging (key-scoped, no users)
A unified framework for LLM routing and evaluation.
A unified interface for multiple LLM providers
A lightweight CLI tool and OpenAI-compatible server for querying multiple Large Language Model (LLM) providers
Package for simple usage of diverse LLMs and configuring them using human readable config files
One-line LLM queries with smart routing and provider failover
A lightweight, production-ready service layer for modular, rate-aware LLM integrations
Multi-provider LLM gateway — one interface, every provider
LLM unified interface