Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 30 of 41

llmstudio-core

LLMStudio core capabilities for routing llm calls for any vendor. No proxy server required. For that use llmstudio[proxy]

llmstudio-proxy

LLMstudio Proxy is the module of LLMstudio that allows calling any LLM as a Service Provider in proxy server. By leveraging LLMstudio Proxy, you can have a centralized point for connecting to any provider running independently from your application.

llmswap

Python SDK, CLI, and Arena for 10 LLM providers. Compare models, synthesize a privacy-aware Best Answer, use MCP tools, fallback, caching, and direct BYOK access.

llmterface-gemini

Gemini provider for LLMterface

llmuri

LLM URI Parsing.

llmux

LLM inference for power users.

llmux-optimizer

Automatically find cheaper LLM alternatives while maintaining performance

llmuxer

Automatically find cheaper LLM alternatives while maintaining performance

llmwire

Lightweight multi-provider LLM client for Python

llmwise

Lightweight Python SDK for the LLMWise multi-model API (chat, compare, blend, judge, failover).

llmx-zk

Added azure GPT to the original foundation

lmos-openai-types

Add your description here

lmux-anthropic

Anthropic provider for lmux

lmux-aws-bedrock

AWS Bedrock provider for lmux

lmux-groq

Groq provider for lmux

lmux-openai

OpenAI provider for lmux

local-openai2anthropic

A lightweight proxy server that converts Anthropic Messages API to OpenAI API

localopenai

Python client library for the OpenAI API

localstack-claude

Claude (Anthropic) API emulator – LocalStack extension

localstack-extension-openai

LocalStack Extension: OpenAI

lollms-client

A client library for LoLLMs generate endpoint

loragateway

lora gateway

mahoraga

Reverse proxy for Python mirrors

majordomo-llm

Unified Python interface for multiple LLM providers with cost tracking

manage-openai
marsgpt

Mars ChatGPT API Lib

matrixllm

MatrixLLM: OpenAI-compatible multi-provider LLM router (OpenRouter-style) with optional relay nodes

maxllm

A unified OpenAI API client with rate limiting, caching, and batch processing

maxllm-gate

maxllm_gate - Intelligent LLM client with built-in rate limiting. Maximizes throughput and prevents 429 errors.

mcp-llm-client-proxy

A Multi-Client Proxy (MCP) for various LLMs with failover and flexible response formats.

mcp-llm-gateway

MCP-compatible LLM gateway that proxies completion requests to downstream OpenAI-compatible providers

megagpt

Unofficial blackbox api library for python

memollm

3-level LLM cache (exact, semantic, prefix) with real-time token cost tracking

meshagent-llm-proxy

Shared LLM proxy helpers for MeshAgent

metis-llm

Metis: local-first AI dev agent. Canonical types, event bus, adapters, routing, tools, memory, sessions, pricing, HTTP/WebSocket server, transparent provider-shape gateway, and CLI/TUI. Installs as `pip install metis-llm`; imported as `metis`.

metorial-anthropic

Anthropic (Claude) provider for Metorial

metorial-mistral

Mistral AI provider for Metorial

metorial-openai

OpenAI provider for Metorial

metorial-openai-compatible

OpenAI-compatible provider base for Metorial

microllama

The smallest possible LLM API

microsoft-openai

A package for creating chat completions with Azure OpenAI.

mikogpt-api

Api for mikogpt.ru service

minimax-tts-openai

A proxy service converts Minimax TTS API to OpenAI-compatible format

mirage-api

Mirage API Python wrapper.

misa-openai

The Python library for the MISA API

mishikallm

Library to easily interface with LLM API providers

mishikallm-proxy-extras

Additional files for the MishikaLLM Proxy. Reduces the size of the main mishikallm package.

mistral-cli-tool

A package to call the Mistral AI API

mistral-dev

Python Client SDK for the Mistral AI API.

mistralai

Python Client SDK for the Mistral AI API.

mitta_mistralai

Fork of MistralAI client library with httpx allowed to be 0.27.0+. Will track with official Mistral release until this is addressed in the main package.

mlgym-ollama-llm

A custom MLGym Ollama wrapper for seamless interactions with LlamaIndex LLMs. This wrapper enables smooth interactions with the Ollama API by automatically including necessary HTTP headers.

mlgym-openai-llm

A custom MLGym OpenAI wrapper for seamless interactions with LlamaIndex LLMs. This wrapper enables smooth interactions with the OpenAI API by automatically including necessary HTTP headers.

mllopenaimanager
model-forge-llm

A reusable library for managing LLM providers, authentication, and model selection.

monollm

A unified framework for accessing multiple LLM providers with comprehensive CLI and machine interface

moonclaude

Moon for Claude: run Claude Code on external LLMs via LiteLLM

mosesaic-llm

Mosesaic LLM layer — provider abstraction, model routing, capability negotiation

multi-llm-orchestrator

Unified interface for Russian LLMs with intelligent routing and fallback

multi-llm-provider

Unified multi-provider LLM interface supporting Claude, OpenAI, Mistral, Ollama, and Clipboard

multillm

VerifAI MultiLLM: A module to invoke multiple LLMs concurrently and rank their results

multillm-core

A unified interface for multiple LLM providers

multiplexer-llm

A multiplexer for Large Language Model APIs built on the OpenAI SDK. It combines quotas from multiple models and automatically uses fallback models when the primary models are rate limited.

muxllm

muxllm is a python library designed to be an all-in-one wrapper for various cloud providers as well as local inference.

mvllm

A FastAPI-based load balancer for vLLM servers with OpenAI-compatible API

myxxxsquared-gpt-invoker

Thin OpenAI Chat Completions invoker with SQLite caching, usage/cost tracking, and reproducible Markdown dumps.

nadir-llm

Dynamic LLM Model Selector based on Prompt Complexity

nanogpt-sdk

Python client for the NanoGPT API

nb-llm-cache

LLMCache is an open-source caching solution designed to operate seamlessly within your cloud infrastructure.

nemollm

Python client library for the NeMo LLM API

neon-llm-core

Core Neon LLM service

netrun-llm

Multi-provider LLM orchestration with fallback chains and three-tier cognition

ngptbot

Wrapper for ChatGPT API

nicegpt

A simple package to make GPT API usage easier

nicellm

NiceLLM is a wrangler of API connections of Large Language Models (LLMs) in Python.

nim-claude-proxy

Anthropic Messages → NVIDIA NIM Proxy for Claude Code

nix-gpt-client

A Python library for easy interaction with the GPT API, featuring typed requests and responses.

noir-llm

A Python package for accessing various LLM models freely

notdiamond-openai

NotDiamond Python library for easy OpenAI integration.

nous-llm

Intelligent No Frills LLM Router - A unified interface for multiple LLM providers

nvd-claude-nim

Anthropic Messages → NVIDIA NIM Proxy for Claude Code

nvd-claude-proxy

Run Claude Code (and any Anthropic SDK client) on NVIDIA NIM models via a local proxy.

oai2ollama

Wrap any OpenAI-compatible API into an Ollama-compatible API.

oci-openai

OCI authentication and authorization utilities for OpenAI python SDK

octoryn-llm

Official Python SDK for the Octoryn Router API

ollama-deproxy

A lightweight, feature-rich proxy for Ollama, designed for development, testing, and staging environments. It simplifies access to remote Ollama instances that are wrapped behind another proxy layer like OpenWebUI. Anthropic and OpenAI compatible endpoints, included.

ollama-herd

Smart multimodal router — LLM inference, image generation, speech-to-text, and embeddings across your device fleet. Cross-platform: macOS, Linux, Windows.

ollama-hydra

Proxy server for managing multiple Ollama models for AutoGen Agents

ollama-openai-proxy

A proxy service that translates Ollama API requests to OpenAI-compatible backends

ollama-proxy

ollama proxy to other cloud model service

omnicomp-claude-oauth

Claude OAuth-based OmniComp provider

omnicomp-claude-re

Claude-RE omnicomp provider

omnicomp-claude-sdk

Claude-agent-sdk omnicomp provider

omnicomp-gemini-oauth

Gemini OAuth-based omnicomp provider

omnicomp-gemini-re

Gemini-RE omnicomp provider

omnicomp-ollama

Ollama omnicomp provider

omnicomp-openai

OpenAI omnicomp provider

omnillm

A unified LLM API utility package.

oneapi2langchain

langchain实现的闭源模型统一API

onellm

A unified interface for interacting with large language models from various providers - a complete drop-in replacement for OpenAI's client

More in Agent frameworks & infrastructure