Agent frameworks & infrastructure

LLM gateways & API proxies

4,018 agents · most starred first

LLM gateways & API proxies, page 31 of 41

onellmclient

A unified Python client to normalize interfaces across major LLM providers (OpenAI, Anthropic, Gemini, DeepSeek, xAI, OpenRouter).

onemo-llm-sdk

基于Linux的一个支持多平台LLM大模型API的SDK客户端

open-batch-llm

A CLI tool to manage batch LLM calls

open-bedrock-server

A unified, provider-agnostic chat completions API server that seamlessly integrates OpenAI and AWS Bedrock through a single endpoint with intelligent format detection and conversion.

open-gemini

Coming soon

open-llm-gateway

A Gateway for LLM API interactions

openai-agent-llm

Customize the LLM interface to make it compatible with the openai interface

openai-aipy

The official Python library for the openai API

openai-api-call

A short wrapper of the OpenAI api call.

openai-api-key-verifier

A Python library for validating OpenAI API keys, and listing available models and verifying model access, and retrieving detailed account usage metrics.

openai-api-manager

This is a simple openAI API wrapper that makes it easier for me (and hopefully for you as well) to send messages via an assistant and manage threads locally with a custom thread key (such as message author) that is associated the thread id stored by open ai.

openai-api-polling

Polling OpenAI API without rate limit issues.

openai-api-server-via-codex

OpenAI-compatible local API server backed by Codex credentials

openai-api-tester

A tool designed to interact with APIs compatible with OpenAI's format.

openai-api-utils

Utilities for OpenAI API Calls

openai-api-wrapper

This Python package provides a comprehensive and efficient wrapper for the OpenAI API, designed to support advanced application development based on ChatGPT.

openai-async

A light-weight, asynchronous client for OpenAI API - text completion, image generation and embeddings.

openai-async-client

OpenAI async API with client side timeout, retry with exponential backoff and connection reuse

openai-async-utils

Async helpers for OpenAI chat

openai-batch

Make OpenAI batch easy to use.

openai-batch-helper

Tiny, production-friendly helper for the OpenAI Batch API

openai-batch-sdk

An SDK for OpenAI ChatGPT batch API for a single process.

openai-batcher

Default template for PDM package

openai-chat-api-wrapper
openai-commands

🛠️ a command interface to the OpenAI API.

openai-express

Max speed OpenAI call. Do not exceed the tpm & rpm limit.

openai-forward

🚀 OpenAI-Forward 是一个专为大型语言模型设计的高级转发代理,提供包括用户请求速率控制、Token速率限制和自定义API密钥等增强功能。该服务可用于代理本地模型和云端模型。OpenAI API Reverse Proxy

openai-http-proxy

OpenAI HTTP Proxy is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

openai-load-balancer

The OpenAI Load Balancer is a Python library designed to distribute API requests across multiple endpoints (supports both OpenAI and Azure OpenAI). It implements a round-robin mechanism for load balancing and includes exponential backoff for retrying failed requests.

openai-local-bridge

Local OpenAI bridge CLI for OpenAI-compatible upstreams

openai-manager

Speed up your OpenAI requests by balancing prompts to multiple API keys.

openai-mealuet

Python library of OpenAI API proxied by mealuet

openai-model-registry

Registry for OpenAI models with capability and parameter validation

openai-model-router

Automatically route your prompts to the best OpenAI model

openai-multi-client

A parallel client for OpenAI API (and more)

openai-oxide

Fastest OpenAI client — Rust core with Python bindings

openai-oxide-python

Fastest OpenAI client — Rust core with Python bindings

openai-parallel-toolkit

OpenAI-Parallel-Toolkit is a Python library for handling multiple OpenAI API keys and parallel tasks. It provides API key rotation, multithreading for faster task execution, and utility functions to boost your OpenAI integration. Ideal for efficient large-scale OpenAI usage.

openai-priority-loadbalancer

A multi-backend, prioritization load balancer for OpenAI

openai-proxy

A price proxy for the OpenAI API

openai-python-cache

A thin wrapper around the OpenAI Python bindings for caching responses.

openai-ratelimiter

A Python module that provides rate limiting capabilities for the OpenAI API, utilizing Redis as a caching service. It helps to manage API usage to avoid exceeding OpenAI's rate limits.

openai-responses-server

CLI to manage the OpenAI Responses Server that bridges chat completions to responses API calls

openai-rest-cli

A full-coverage CLI for the OpenAI REST API

openai-router

openai-router

openai-to-codex-wrapper

CLI to manage the OpenAI to Codex Wrapper API server

openai-unofficial

Free & Unlimited Unofficial OpenAI API Python SDK - Supports all OpenAI Models & Endpoints

openai-utilities

Wrapper on top of openai python SDK primarily to interface with openai and anyscale public endpoints.

openai-wrapi

OpenAI API proxy for fine-grained cost tracking & control and caching of responses

openai2claude

Proxy Claude API requests to OpenAI-compatible backends

openai_request_runner

Slim Runner for batched OpenAI Requests

openaiapi

A light wrapper around the OpenAI API using apiron

openaicustomiz

Python client library for the OpenAI API

openaidantic

A fully typed user friendly openai wrapper package

openaiflow

A Python wrapper for OpenAI API interactions.

openaiinsight

Updated the depreciating OpenAI model.

openaiir

OpenAIIR → This package is now `aiir`. Install: pip install aiir

openaijsonwrapper

小型 JSON 封装,适配 OpenAI 风格的数据与自定义类型序列化,支持多模态图片输入

openaipro

Python client library for the OpenAI API

openapiopenai

OpenAI API

opengemini-client

A Python package for interacting with the opengemini

opengig_openai_wrapper

easy to use openai apis

opengpt4

OpenGPT 3.5/4 is a project aimed at providing practical and user-friendly APIs. The APIs allow for easy integration with various applications, making it simple for developers to incorporate the natural language processing capabilities of GPT into their projects.

openrouter-agent

This package provides integration between Pydantic AI and OpenRouter, allowing you to use any model available on OpenRouter with the Pydantic AI framework. The implementation leverages the OpenAI-compatible API provided by OpenRouter.

openrun-llm

Turn any local AI model into an OpenAI-compatible API

opsmate-provider-groq

Groq provider for opsmate

optillm

An optimizing inference proxy for LLMs.

opyngpt

Supercharge your projects with LLMs. No API keys required.

orchestra-llm-cache

Reduce AI orchestration costs by 85% with semantic caching

pandasai-bedrock

AWS Bedrock integration for PandaAI

pandasai-litellm

LiteLLM integration for PandaAI

pandasai-openai

OpenAI integration for PandasAI

pangea-anthropic

A wrapper around the Anthropic Python library

pangea-llm-translator

A library to transform messages for different LLMs like OpenAI, Mistral, LangChain, Gemini, DBRX, Cohere, Claude, and AWS Bedrock to Pangea messages format

parallm

CLI tool for querying multiple models with prompts from a CSV with schema support

pennsieve-llm

Thin configuration helper for the Pennsieve LLM Governor — returns a pre-configured anthropic.Anthropic client.

pgptAI

All in one AIGC API package for Python

pi-llm

Unified LLM provider abstraction for Python — streaming, tool calling, extended thinking, cost tracking

pi-llm-server

统一 LLM 服务网关 - 集成 Embedding、ASR、Reranker、OCR 服务

pickle-openai

Securely Dumping and Loading an OpenAI Client with Pickle

pig-llm

Unified multi-provider LLM API (OpenAI, Anthropic, Google, etc.)

pinellm

A Python library for managing LLM chat models

pixelarrayllm

PixelArray LLM微服务客户端

pixigpt

Production-grade Python client for the PixiGPT API

plugllm

Unified LLM API interface for OpenAI, Gemini, Mistral, Groq, Claude, and 10+ providers

pocketgroq

A library for easy integration with Groq API, including web scraping, image handling, and Chain of Thought reasoning

polar-llama

Parallel inference calls to LLM APIs using Polars dataframes with Pydantic-based structured outputs

polars-llm

Call LLMs, decision models, and embedding models from a Polars DataFrame using native Polars expressions.

polyllm

A unified interface for querying multiple large language models

pplx-python-sdks-llm-api
process-gpt-llm-factory

통합 LLM 팩토리 라이브러리 - 다양한 LLM 제공자를 통합된 인터페이스로 사용

prollama

Intelligent LLM Execution Layer for developer teams — anonymize code, route models, solve tickets.

prollama-dev

Intelligent LLM Execution Layer for developer teams — anonymize code, route models, solve tickets.

proxyllm

LLM Proxy to reduce cost and complexity of using multiple LLMs

pulse-anthropic

Anthropic Claude adapter for PULSE Protocol — send PULSE messages, get Claude responses

pulse-gemini

Google Gemini adapter for PULSE Protocol — send PULSE messages, get Gemini responses

pulse-ollama

Ollama adapter for PULSE Protocol — run local AI models with the same interface as cloud providers

pulse-openai

OpenAI adapter for PULSE Protocol — send PULSE messages, get AI responses

putkoff-chatGPT-API

`putkoff_chatGPT_API` is a Python module for interacting with OpenAI's GPT models. It simplifies the process of making API calls, managing API keys, and parsing responses, while also providing utility functions to work with timestamps and organize response data.

py-context-llm7

Minimal Python client for https://api.context.llm7.io

More in Agent frameworks & infrastructure