Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 27 of 41

llama-index-llms-together

llama-index llms together integration

llama-index-llms-unify

llama-index llms unify integration

llama-index-llms-unionllm

A custom LlamaIndex Llms integration for UnionLLM

llama-index-llms-upstage

llama-index llms upstage integration

llama-index-llms-vercel-ai-gateway

llama-index llms vercel ai gateway integration

llama-index-llms-vertex

llama-index llms vertex integration

llama-index-llms-watsonx

llama-index llms watsonx integration

llama-index-llms-yi

llama-index llms yi integration

llama-index-llms-you

llama-index LLM integration with You.com's conversational Smart and Research APIs

llama-index-llms-zhipuai

llama-index llms zhipuai integration

llama-index-selectors-notdiamond

llama-index selectors Not Diamond integration

llama-integration

Unified interface for multiple LLM providers with advanced features (routing, privacy, logging).

llama-proxy

Proxy service for the LlamaAI ecosystem.

llama_cpp_openai

Lightweight OpenAI API server on top of Llama local models. Suitable for combined use with Microsoft AutoGen.

llamaai

Llama API Python Client

llamaapi

Llama API python SDK

llamaapi-llamasearch

A flexible API client and server utilities package

llamakey

One master key for all LLM/GenAI endpoints

llamapass

CLI client for LlamaPass - Ollama gateway with authentication and token tracking

llm

CLI utility and Python library for interacting with Large Language Models from organizations like OpenAI, Anthropic and Gemini plus local models installed on your own machine.

llm-1min

LLM access to models served by 1minAI

llm-1minai

LLM plugin providing access to 30 curated frontier AI models through the 1min.ai API (chat-with-ai endpoint) with SSE streaming, attachments, AI memory, brand voice, web search, and persistent options

llm-acuration

An LLM package made by Acuration

llm-adapter-gen-ai-proj

Generic abstraction layer for GenAI services

llm-adapters

Adapters as API gateways to Different LLM Models

llm-adaptive-router

An adaptive router for LLM model selection

llm-ads

FastAPI middleware for LLM ad serving integration

llm-aggregator

A model aggregator service for multiple LLM backends.

llm-agregator

A package for generating diagrams using various models

llm-all-models-async

Make all LLM sync models available as async

llm-anthropic

LLM access to models by Anthropic, including the Claude series

llm-anygate

A flexible gateway for connecting and managing multiple LLM providers

llm-anyscale-endpoints

LLM plugin for models hosted by Anyscale Endpoints

llm-api-access

Unified interface to query popular LLM providers via PyO3

llm-api-adapter

Lightweight multi-provider LLM API adapter for Python — OpenAI, Anthropic, Google, Mistral, xAI, Qwen, Kimi, DeepSeek, and Z.ai.

llm-api-bridge

A unified interface for querying multiple LLM providers via REST APIs

llm-api-client

A client for interacting with LLM completion APIs and tracking usage.

llm-api-engine

Unified LLM API provider and engine library

llm-api-multiple

The official Python library for the llm_api API

llm-api-open

Unofficial open APIs for popular LLMs with self-hosted redirect capability

llm-api-python

A python lib to call LLM API models.

llm-api-router

A unified API routing library for Large Language Models

llm-api-server

LLM plugin to expose a FastAPI server with compatible APIs for popular LLM clients

llm-async

Multi-LLM Provider Library

llm-azure

Azure AI Foundry plugin for LLM - access Anthropic, Llama, Mistral and more via Azure

llm-azure-ai

Add your description here

llm-azure-ai-foundry

LLM plugin to access model deployments on Azure AI Foundry and Foundry Local

llm-azure-openai

LLM plugin to access Azure OpenAI models

llm-base

LLM Base

llm-bedrock

Run prompts against models hosted on AWS Bedrock

llm-bedrock-amazon

LLM plugin for Amazon Titan on AWS Bedrock

llm-bedrock-anthropic

LLM plugin for Anthropic's Claude on AWS Bedrock

llm-bedrock-converse

LLM plugin for AWS Bedrock Converse API with tool calling and embeddings support

llm-bedrock-meta

LLM plugin for Meta Llama2 on AWS Bedrock

llm-bedrock-mistral

LLM plugin for Mistral on AWS Bedrock

llm-bedrock-nova

LLM plugin for Amazon's Nova on AWS Bedrock

llm-blanket

Unified Python library for LLM APIs (OpenAI, Anthropic, Gemini, xAI, Groq, custom)

llm-cache-router

Semantic cache, multi-provider LLM router and cost tracker (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen)

llm-caching

Cache LLM API calls to speed up development and prevent rate limits

llm-caiche

Unified Language Model Cache.

llm-call-msagent

Simple Azure OpenAI call helper

llm-cat

llm api and prompt

llm-cerebras

Plugin for LLM adding fast Cerebras inference API support

llm-chat-openapi

chat api collection for popular LLM

llm-chutes

LLM plugin for models hosted by Chutes AI

llm-claude

LLM plugin for Anthropic's Claude

llm-claude-3

llm-claude-3 is now llm-anthropic

llm-cli-gateway

llm plugin for multi-LLM orchestration via llm-cli-gateway

llm-cli-token-heimdall

Universal AI CLI Proxy, Firewall, and Token Intelligence Gateway

llm-client-redis

LLM redis中介的客户端程序

llm-cloudflare

An LLM CLI plugin for Cloudflare Workers AI models.

llm-cohere

Plugin for LLM adding support for Cohere's Generate and Summarize models

llm-command-r

Access the Cohere Command R family of models

llm-commons

A Python wrapper for managing OpenAI API and other LLM models.

llm-completion

A library for LLM text completion using LiteLLM

llm-conduit

A provider-agnostic SDK for LLM, image, video, and embedding model clients

llm-connect

LLM Connect API

llm-connector

A unified Python connector for multiple LLM providers with YAML-driven configuration.

llm-connectors

A library of connectors for popular AI platforms

llm-cookie-bridge

Unified async cookie/session bridge for major AI web apps.

llm-cost-optimizer

Automatically reduce LLM API costs. Routes to cheapest model that succeeds. Use kalibr instead.

llm-cost-reduction

Reduce LLM costs automatically with outcome-based routing. Use kalibr instead.

llm-custom-call
llm-deepseek

Access deepseek.com models via API

llm-deepseek-xtreme

LLM access to DeepSeek's API

llm-deepseek-ya

Yet another LLM access to DeepSeek's API

llm-dial

A minimal, auditable Python library for calling multiple LLM providers. Lightweight LiteLLM alternative.

llm-digitalocean

LLM plugin for models hosted by DigitalOcean AI

llm-discovery

LLM model discovery and tracking system for real-time monitoring of available models across multiple providers

llm-dispatcher

Intelligent LLM dispatching with performance-based routing, multimodal support, streaming, monitoring, and comprehensive analytics

llm-duckduckgo-chat

LLM plugin for Duckduckgo AI Chat service

llm-electronhub

LLM plugin for models hosted by ElectronHub

llm-engines

A unified inference engine for large language models (LLMs) including open-source models (VLLM, SGLang, Together) and commercial models (OpenAI, Mistral, Claude).

llm-ensemble

Create ensembles of Large Language Models to query them all at the same time.

llm-factory

This package is a utility library for interfacing with multiple Large Language Models (LLMs) using a common interface.

llm-factory-tag

Een package om LLM modellen te initialiseren op basis van provider.

llm-factory-toolkit

A toolkit for interacting with various Large Language Models

llm-failover

Multi-LLM provider client with automatic failover and priority ordering

llm-fallback

Automatic failover between LLM providers. When OpenAI is down, seamlessly switch to Anthropic, Google, or any backup.

llm-fallbacks

A comprehensive Python library for managing fallback mechanisms for Large Language Model (LLM) API calls using LiteLLM

More in Agent frameworks & infrastructure