Business agents

Semantic cache, multi-provider LLM router and cost tracker (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen)

Lightweight caching layer for LLM API responses with TTL and size limits

LLM plugin for Anthropic's Claude

llm-claude-3 is now llm-anthropic

Production-grade LLMOps infrastructure for context window management, token counting, document chunking, and compression

Composable building blocks for LLM context engineering

Cost tracking for OpenAI, Gemini, and Claude APIs with session management

Real-time cost monitoring and budget enforcement for LLM API calls

Python SDK for LLM Cost Monitor - Track, aggregate, and analyze LLM usage costs

Automatically reduce LLM API costs. Routes to cheapest model that succeeds. Use kalibr instead.

Reduce LLM costs automatically with outcome-based routing. Use kalibr instead.

Track LLM API costs per request. Know where your tokens go.

A minimal, auditable Python library for calling multiple LLM providers. Lightweight LiteLLM alternative.

LLM model discovery and tracking system for real-time monitoring of available models across multiple providers

Carbon-aware scheduling and telemetry for LLM applications

Create ensembles of Large Language Models to query them all at the same time.

CLI tool to expose LLMs through messaging clients (Telegram, Discord, Slack, etc.)

Multi-LLM provider client with automatic failover and priority ordering

Automatic failover between LLM providers. When OpenAI is down, seamlessly switch to Anthropic, Google, or any backup.

LLM plugin for analyzing FEC campaign finance filings

A thin, fast, observable Python client for LLMs

AI-powered filesystem cleanup tool with interactive CLI

Run URLs through the pure.md API

Run URLs through the Jina Reader API

A plugin for llm to load Reddit threads as fragments

Run URLs through the Trafilatura to get plain text

A unified handler for LLM API calls across OpenAI, Anthropic, and Google

CLI-based threat hunting platform using LLM and RAG for semantic search of security logs

Drop-in prompt injection defense for LLM apps and AI agents — detect, sanitize, block, and audit injection attacks in real time. Includes multi-turn session scanning, allow-lists, rate-abuse detection, multi-layer scanner, FastAPI and Flask middleware.

A Python library for managing multi-agent model invocation with automatic failover strategies

A unified interface for streaming structured JSON from OpenAI, Anthropic, and Google Gemini.

One tiny model, every LLM API. Drop-in test server for OpenAI, Anthropic, Bedrock, and Vertex.

A unified toolkit for working with multiple LLM providers

Cost ceiling, audit log, and kill switch for LLM agents.

A thread-safe and async rate limiter for Gemini and OpenAI models.

Lightweight LLM call logger for OpenAI/Anthropic

Stable canonical sha256 hash of LLM request/message structures. Recursive key-sorted JSON canonicalization with per-provider presets that drop noise fields. For cache keys and idempotency. Zero runtime deps.

pytest plugin to mock and replay OpenAI and Anthropic API calls — record once, replay forever, no API key needed

Track and check deprecation status of LLM provider models (OpenAI, Anthropic, etc.)

Compare LLM model outputs side-by-side with rich diff visualization

Intelligent multi-LLM request router for cost optimization

Mutation testing for LLM prompts. Find the gaps in your eval suite before production does.

Lightweight SDK for LLM inference logging and observability

Production-ready LLM platform building blocks — observability, prompts, eval, RAG, hallucination guards

Local preflight checks for LLM provider keys, endpoints, and model configuration

Reliable, structured, production-safe LLM outputs with schema validation and auto-repair

A unified interface for multiple LLM providers

LLM Proxy Server is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

Real-time observability dashboard for LLM applications. Track prompts, tokens, costs, and latency. One-line integration.

MCP server providing real-time AI model intelligence - pricing, capabilities, and recommendations

Cloud-agnostic rate limit mitigation for LLM APIs

A centralized registry for discovering and managing LLM model capabilities. Track model features, costs, and limitations across providers like OpenAI and Anthropic. Supports both verified model definitions and user-managed entries with local storage.

SQLite-backed LLM response cache. Exact match + fuzzy match. Decorator API. Zero mandatory server dependencies.

LLM-Rosetta: Unified LLM API adapter — bidirectional conversion between OpenAI, Anthropic, and Google formats via intermediate representation (IR).

Fault-tolerant LLM provider rotation with circuit breaker, quotas, and model-first routing

Intelligent LLM routing with weighted least-outstanding selection, 429 cooldown, and request class isolation

Route LLM calls through one send_message() across OpenAI-compatible providers and Anthropic's native Messages API; keep a JSONL ledger of every request and response for offline cost reconciliation.

DEPRECATED — use pii-veil + pii-core instead. Reversible PII anonymization for Polish documents.

A Python library for selecting the best LLM model based on user input using any LLM via LiteLLM

Multi-provider AI chat interface supporting 9+ LLM providers (Anthropic, OpenAI, xAI, Cohere, Mistral, Perplexity, Gemini, HuggingFace, Groq)

輕量級 LLM 模型追蹤與自動替換 Decorator

Track and limit LLM API spending in real-time. Drop-in middleware for OpenAI, Anthropic, Google, and any OpenAI-compatible API.

A comprehensive LLM tracing and monitoring library

Agent framework with tool calling for OpenAI, Anthropic, and Google

Python library for standardization, optimization and monitoring of LLM API calls

Pass text through a chain of 20 LLM translations and watch the meaning drift

Deterministic testing framework for LLM-powered apps — record/replay cassettes, eval scoring, regression testing

Token-bucket rate limiter for LLM API calls with per-model limits

Pre-call cost estimation, tracking, and budgets for OpenAI, Gemini, and Claude

Token cost control and auto-optimization for LLM apps — compress prompts, estimate costs, enforce budgets, route to cheap models, and cut LLM spend by up to 60%

Cut your LLM API bill by 30-70% with zero accuracy loss

A lightweight Python library for tracking OpenAI and Anthropic SDK costs with budget alerts

Lightweight drop-in Python decorator to track costs, monitor token usage, and enforce budget/rate limits for LLM API calls

OpenTelemetry instrumentation for Anthropic.

Auto-capture OpenAI and Anthropic LLM calls for warehousing and observability

LLM-powered markdown knowledge base CLI — knowledge synthesized at ingest time, not query time.

Unified Python client for OpenAI, Azure OpenAI, Vertex AI, Anthropic, Gemini, DeepSeek, OpenRouter, Cerebras, Fireworks, Together AI, LM Studio, Bedrock, LiteLLM, and ChatGPT.

A zero‑dependency wrapper that turns any OpenAI‑compatible endpoint into a hackable one‑liner.

Lightweight LLM API Orchestrator - A fast, multi-provider LLM client with rate limiting and key rotation

The new package transforms a user’s plain‑text search phrase into a structured, BM25‑ranked list of PostgreSQL search results. It takes the query text as input, forwards it to an LLM that rewrites the

Unified Python interface for OpenAI, Anthropic, Google, and Ollama LLMs with optional model registry and usage tracking - deprecated

A framework-agnostic browser tool for AI agents. Works with Anthropic, OpenAI, LangChain, and any agent framework.

Real-time LLM spend enforcement — CLI and tray app

A professional CLI tool for interacting with OpenAI and Anthropic LLMs with cost tracking

Cycle through multiple LLM providers with smart fallback, load balancing, and unified API with real-time FastAPI dashboard, resilient streaming failover, Pydantic structured output, prompt caching, token budgets, and local/cloud key rotation.

Read through cache for OpenAI's API.

Secure LLM Mesh over MCP — multi-LLM swarm + industrial IoT + privacy gateway

A lightweight, cross-platform latency and throughput profiler for LLMs

In-process LLM client faker that patches OpenAI, Anthropic, LiteLLM, and LangChain for fast, deterministic testing without network calls

Lightweight, provider-agnostic LLM calling library — OpenAI, Gemini, Anthropic, Groq, Mistral, Cohere, Azure, Bedrock, Ollama

LLMGovernor — anomaly engine for LLM apps (core SDK with optional LiteLLM bundle)

Open-source LLM evaluation framework — 50+ metrics for RAG, agents, safety, async eval, regression tracking, custom benchmarks, and exportable reports

Plug-and-play LLM connector via YAML config

Your Personal AI Cloud -- intelligent proxy, router, and cache for LLMs

LLM execution engine — multi-provider streaming, MCP tools, and budget management

Production-grade LLM client for Python - 100+ providers, 11,000+ models. Rust-powered.

AI API gateway SDK with cost tracking, budget enforcement, and multi-provider routing for LLM applications

Unified LLM testing lab - test, compare, and interact with multiple AI providers through a single UI

Reduce LLM costs by 90% - AI recommendations with NO API keys needed!

A unified framework for interacting with various LLM providers through a consistent interface with batch processing, templating, and rate limiting.