Agent frameworks & infrastructure
LLM gateways & API proxies
4,018 agents · most starred first
LLM gateways & API proxies, page 30 of 41
LLMStudio core capabilities for routing llm calls for any vendor. No proxy server required. For that use llmstudio[proxy]
LLMstudio Proxy is the module of LLMstudio that allows calling any LLM as a Service Provider in proxy server. By leveraging LLMstudio Proxy, you can have a centralized point for connecting to any provider running independently from your application.
Python SDK, CLI, and Arena for 10 LLM providers. Compare models, synthesize a privacy-aware Best Answer, use MCP tools, fallback, caching, and direct BYOK access.
Gemini provider for LLMterface
LLM URI Parsing.
LLM inference for power users.
Automatically find cheaper LLM alternatives while maintaining performance
Automatically find cheaper LLM alternatives while maintaining performance
Lightweight multi-provider LLM client for Python
Lightweight Python SDK for the LLMWise multi-model API (chat, compare, blend, judge, failover).
Added azure GPT to the original foundation
Add your description here
Anthropic provider for lmux
AWS Bedrock provider for lmux
Groq provider for lmux
OpenAI provider for lmux
A lightweight proxy server that converts Anthropic Messages API to OpenAI API
Python client library for the OpenAI API
Claude (Anthropic) API emulator – LocalStack extension
LocalStack Extension: OpenAI
A client library for LoLLMs generate endpoint
lora gateway
Reverse proxy for Python mirrors
Unified Python interface for multiple LLM providers with cost tracking
Mars ChatGPT API Lib
MatrixLLM: OpenAI-compatible multi-provider LLM router (OpenRouter-style) with optional relay nodes
A unified OpenAI API client with rate limiting, caching, and batch processing
maxllm_gate - Intelligent LLM client with built-in rate limiting. Maximizes throughput and prevents 429 errors.
A Multi-Client Proxy (MCP) for various LLMs with failover and flexible response formats.
MCP-compatible LLM gateway that proxies completion requests to downstream OpenAI-compatible providers
Unofficial blackbox api library for python
3-level LLM cache (exact, semantic, prefix) with real-time token cost tracking
Shared LLM proxy helpers for MeshAgent
Metis: local-first AI dev agent. Canonical types, event bus, adapters, routing, tools, memory, sessions, pricing, HTTP/WebSocket server, transparent provider-shape gateway, and CLI/TUI. Installs as `pip install metis-llm`; imported as `metis`.
Anthropic (Claude) provider for Metorial
Mistral AI provider for Metorial
OpenAI provider for Metorial
OpenAI-compatible provider base for Metorial
The smallest possible LLM API
A package for creating chat completions with Azure OpenAI.
Api for mikogpt.ru service
A proxy service converts Minimax TTS API to OpenAI-compatible format
Mirage API Python wrapper.
The Python library for the MISA API
Library to easily interface with LLM API providers
Additional files for the MishikaLLM Proxy. Reduces the size of the main mishikallm package.
A package to call the Mistral AI API
Python Client SDK for the Mistral AI API.
Python Client SDK for the Mistral AI API.
Fork of MistralAI client library with httpx allowed to be 0.27.0+. Will track with official Mistral release until this is addressed in the main package.
A custom MLGym Ollama wrapper for seamless interactions with LlamaIndex LLMs. This wrapper enables smooth interactions with the Ollama API by automatically including necessary HTTP headers.
A custom MLGym OpenAI wrapper for seamless interactions with LlamaIndex LLMs. This wrapper enables smooth interactions with the OpenAI API by automatically including necessary HTTP headers.
A reusable library for managing LLM providers, authentication, and model selection.
A unified framework for accessing multiple LLM providers with comprehensive CLI and machine interface
Moon for Claude: run Claude Code on external LLMs via LiteLLM
Mosesaic LLM layer — provider abstraction, model routing, capability negotiation
Unified interface for Russian LLMs with intelligent routing and fallback
Unified multi-provider LLM interface supporting Claude, OpenAI, Mistral, Ollama, and Clipboard
VerifAI MultiLLM: A module to invoke multiple LLMs concurrently and rank their results
A unified interface for multiple LLM providers
A multiplexer for Large Language Model APIs built on the OpenAI SDK. It combines quotas from multiple models and automatically uses fallback models when the primary models are rate limited.
muxllm is a python library designed to be an all-in-one wrapper for various cloud providers as well as local inference.
A FastAPI-based load balancer for vLLM servers with OpenAI-compatible API
Thin OpenAI Chat Completions invoker with SQLite caching, usage/cost tracking, and reproducible Markdown dumps.
Dynamic LLM Model Selector based on Prompt Complexity
Python client for the NanoGPT API
LLMCache is an open-source caching solution designed to operate seamlessly within your cloud infrastructure.
Python client library for the NeMo LLM API
Core Neon LLM service
Multi-provider LLM orchestration with fallback chains and three-tier cognition
Wrapper for ChatGPT API
A simple package to make GPT API usage easier
NiceLLM is a wrangler of API connections of Large Language Models (LLMs) in Python.
Anthropic Messages → NVIDIA NIM Proxy for Claude Code
A Python library for easy interaction with the GPT API, featuring typed requests and responses.
A Python package for accessing various LLM models freely
NotDiamond Python library for easy OpenAI integration.
Intelligent No Frills LLM Router - A unified interface for multiple LLM providers
Anthropic Messages → NVIDIA NIM Proxy for Claude Code
Run Claude Code (and any Anthropic SDK client) on NVIDIA NIM models via a local proxy.
Wrap any OpenAI-compatible API into an Ollama-compatible API.
OCI authentication and authorization utilities for OpenAI python SDK
Official Python SDK for the Octoryn Router API
A lightweight, feature-rich proxy for Ollama, designed for development, testing, and staging environments. It simplifies access to remote Ollama instances that are wrapped behind another proxy layer like OpenWebUI. Anthropic and OpenAI compatible endpoints, included.
Smart multimodal router — LLM inference, image generation, speech-to-text, and embeddings across your device fleet. Cross-platform: macOS, Linux, Windows.
Proxy server for managing multiple Ollama models for AutoGen Agents
A proxy service that translates Ollama API requests to OpenAI-compatible backends
ollama proxy to other cloud model service
Claude OAuth-based OmniComp provider
Claude-RE omnicomp provider
Claude-agent-sdk omnicomp provider
Gemini OAuth-based omnicomp provider
Gemini-RE omnicomp provider
Ollama omnicomp provider
OpenAI omnicomp provider
A unified LLM API utility package.
langchain实现的闭源模型统一API
A unified interface for interacting with large language models from various providers - a complete drop-in replacement for OpenAI's client