Openai agents

LLM Application Performance Monitoring - Real-time monitoring for LLM-powered applications

LLM-App is a library for creating responsive AI applications leveraging OpenAI/Hugging Face APIs to provide responses to user queries based on live data sources. Build your own LLM application in 30 lines of code, no vector database required.

Multi-LLM Provider Library

Automatic micro-batching for HTTP LLM calls and local PyTorch inference, backed by a Rust core.

39% faster TTFT, 67% less KV cache, zero config — autotune optimises local LLMs on Ollama, LM Studio, and MLX

LLM plugin to access Azure OpenAI models

Text-to-speech using the Azure OpenAI TTS API

Batch-oriented LLM annotation workflows for tabular datasets with OpenAI Batch support.

A package to process CSV text data in batches using OpenAI API

A developer-centric CLI tool to systematically evaluate and compare Large Language Models (LLMs)

CLI tool for comparing LLM API pricing, ranked by cost-effectiveness against LMSYS Arena scores

LLM Inference Benchmark CLI - measure TTFT, TPS, ITL, E2E latency for any OpenAI-compatible API

Benchmark LLMs with 10 benchmarks & 132K+ questions. 8 providers: OpenAI, Anthropic, Groq, Together, Fireworks, DeepSeek, Ollama, HuggingFace. Unified CLI + Web dashboard.

Benchmark any LLM provider against your actual prompts — latency, cost, quality

Unified Python library for LLM APIs (OpenAI, Anthropic, Gemini, xAI, Groq, custom)

Pre-flight LLM cost estimation and budget enforcement

Agent Cache Runtime for building cache-aware LLM applications.

Local SQLite cache for OpenAI and Anthropic API calls. One env var, 60-80% cheaper dev loops.

Semantic cache, multi-provider LLM router and cost tracker (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen)

Lightweight caching layer for LLM API responses with TTL and size limits

Cache LLM API calls to speed up development and prevent rate limits

Simple Azure OpenAI call helper

Calculate CO2 emissions from LLM API calls following Green Software Foundation standards.

A simple CI/CD utility for running LLM tasks with Semantic Kernel

An OpenAI LLM based CLI coding assistant.

A Python wrapper for managing OpenAI API and other LLM models.

Command-line interface for LLMs with advanced features like tool calling, file handling, and more.

Production-grade LLMOps infrastructure for context window management, token counting, document chunking, and compression

Composable building blocks for LLM context engineering

Portable, model-agnostic memory layer for LLM conversations

Cost tracking for OpenAI, Gemini, and Claude APIs with session management

Estimate LLM request cost and enforce per-request or per-session budgets. Python port of @mukundakatta/llm-cost-guard.

Real-time cost monitoring and budget enforcement for LLM API calls

Python SDK for LLM Cost Monitor - Track, aggregate, and analyze LLM usage costs

Automatically reduce LLM API costs. Routes to cheapest model that succeeds. Use kalibr instead.

Reduce LLM costs automatically with outcome-based routing. Use kalibr instead.

Track LLM API costs per request. Know where your tokens go.

A minimal, auditable Python library for calling multiple LLM providers. Lightweight LiteLLM alternative.

A CLI tool for comparing LLM outputs — semantically, visually, and at scale

LLM model discovery and tracking system for real-time monitoring of available models across multiple providers

Intelligent LLM dispatching with performance-based routing, multimodal support, streaming, monitoring, and comprehensive analytics

Model distiller automator — recursively drives an LLM with seed prompts and stores compressed outputs in SQLite

AI-powered documentation generator for code projects

A unified inference engine for large language models (LLMs) including open-source models (VLLM, SGLang, Together) and commercial models (OpenAI, Mistral, Claude).

Create ensembles of Large Language Models to query them all at the same time.

Drop-in observability for LLM applications — quality scoring, hallucination detection, cost tracking, and agent run debugging.

Advanced Knowledge Graph Engine with Document Processing, Semantic Search and Multi-LLM Integration

LLM Expect is a minimalist, developer-first SDK for testing LLM-powered Python functions using structured JSONL datasets.

Multi-LLM provider client with automatic failover and priority ordering

Automatic failover between LLM providers. When OpenAI is down, seamlessly switch to Anthropic, Google, or any backup.

Automated feature engineering using Large Language Models (LLMs) for tabular data

A thin, fast, observable Python client for LLMs

AI-powered filesystem cleanup tool with interactive CLI

Small CLI playground for Qwen via an OpenAI-compatible LLM Forge gateway.

A configurable forwarder for OpenAI-compatible LLM requests

A lightweight Python library for creating reliable, contract-driven LLM functions whose core logic is implemented by an LLM.

A unified handler for LLM API calls across OpenAI, Anthropic, and Google

OpenAI-compatible inference server: Llama 3.1 8B + Whisper + Kokoro TTS exposed via ngrok

HTTP API for LLM with OpenAI compatibility

Drop-in prompt injection defense for LLM apps and AI agents — detect, sanitize, block, and audit injection attacks in real time. Includes multi-turn session scanning, allow-lists, rate-abuse detection, multi-layer scanner, FastAPI and Flask middleware.

A Python library for managing multi-agent model invocation with automatic failover strategies

A lightweight Model I/O normalization layer for OpenAI-compatible LLM calls.

A unified interface for streaming structured JSON from OpenAI, Anthropic, and Google Gemini.

One tiny model, every LLM API. Drop-in test server for OpenAI, Anthropic, Bedrock, and Vertex.

A unified toolkit for working with multiple LLM providers

A thread-safe and async rate limiter for Gemini and OpenAI models.

LLM plugin for LiteLLM OpenAI-compatible proxies

Lightweight LLM call logger for OpenAI/Anthropic

Mask sensitive data in documents using a local OpenAI-compatible LLM

Stable canonical sha256 hash of LLM request/message structures. Recursive key-sorted JSON canonicalization with per-provider presets that drop noise fields. For cache keys and idempotency. Zero runtime deps.

Accurate LLM usage & cost tracking for Python backends (FastAPI-native)

pytest plugin to mock and replay OpenAI and Anthropic API calls — record once, replay forever, no API key needed

Track and check deprecation status of LLM provider models (OpenAI, Anthropic, etc.)

Compare LLM model outputs side-by-side with rich diff visualization

OpenAI-compatible API server for simonw's llm cli

Intelligent multi-LLM request router for cost optimization

Lightweight SDK for LLM inference logging and observability

OpenAI-compatible LLM proxy with SQLite request capture, observability, and an admin UI.

Access OpenAI models via a Codex subscription

LLM plugin for OpenAI and Google image generation and editing

LLM plugin for OpenAI

Access OpenAI models via a Codex subscription

Drop-in OpenTelemetry GenAI observability for any LLM backend — local or cloud.

Talk to your PDFs using an LLM.

Benchmark the performance (output speed, latency) of OpenAI compatible endpoints

Production-ready LLM platform building blocks — observability, prompts, eval, RAG, hallucination guards

Local preflight checks for LLM provider keys, endpoints, and model configuration

A generic JSON prettier and interactive HTML viewer, built especially for parsing embedded JSON from LLM payloads and responses.

Reliable, structured, production-safe LLM outputs with schema validation and auto-repair

A unified interface for multiple LLM providers

A unified interface for multiple LLM providers with image generation, speech-to-text, and function calling (OpenAI, Anthropic, Gemini, VertexAI, Ollama + DALL-E, Replicate, Whisper, Google Speech, Tools)

LLM Proxy Server is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

Real-time observability dashboard for LLM applications. Track prompts, tokens, costs, and latency. One-line integration.

MCP server providing real-time AI model intelligence - pricing, capabilities, and recommendations

Cloud-agnostic rate limit mitigation for LLM APIs

Transform any LLM into a methodical thinker that excels at systematic reasoning like OpenAI o1 and DeepSeek R1

A Python framework for building intelligent, recursive task decomposition systems powered by Large Language Models

A centralized registry for discovering and managing LLM model capabilities. Track model features, costs, and limitations across providers like OpenAI and Anthropic. Supports both verified model definitions and user-managed entries with local storage.

A flexible library for testing LLM responses against predefined rubrics using OpenAI's API for automated scoring

Unified LLM usage management — API proxy, session diagnostics, multi-CLI orchestration.