Business agents

LlamaIndex LLM integration for Ferro Labs AI Gateway — route LlamaIndex workloads across 30+ LLM providers via a single OpenAI-compatible endpoint

llama-index multi-modal-llms anthropic integration

llama-index packs chroma_autoretrieval integration

llama-index packs implementation of a hierarchical agent for finance chat.

llama-index postprocessor FlashRank rerank integration

llama-index readers airbyte_salesforce integration

llama-index readers chroma integration

llama-index readers earnings_call_transcript integration

llama-index readers sec_filings integration

LlamaIndex integration for BuyWhere — search products, discover deals, and browse categories through the BuyWhere product catalog API

llama-index finance-tools integration

OSINT intelligence platform integration for LlamaIndex — 15-category intelligence reports, domain recon, and social media lookup

llama-index tools salesforce integration

llama-index tools yahoo_finance integration

llama-index vector_stores chroma integration

Unified interface for multiple LLM providers with advanced features (routing, privacy, logging).

llama-optimus is a lightweight Python tool to automatically optimize llama.cpp performance flags for maximum tg & pp token/s throughput. Powered by Bayesian optimization with Optuna

Asynchronous Fortnite API wrapper.

CLI utility and Python library for interacting with Large Language Models from organizations like OpenAI, Anthropic and Gemini plus local models installed on your own machine.

LLM actor: pool, circuit breaker, retries, metrics

LLM access to models by Anthropic, including the Claude series

A flexible gateway for connecting and managing multiple LLM providers

Lightweight multi-provider LLM API adapter for Python — OpenAI, Anthropic, Google, Mistral, xAI, Qwen, Kimi, DeepSeek, and Z.ai.

Runs a throughput benchmark for LLM APIs, measuring generation throughput, prompt throughput, and Time To First Token (TTFT) under various concurrency levels.

Multi-LLM Provider Library

Azure AI Foundry plugin for LLM - access Anthropic, Llama, Mistral and more via Azure

LLM plugin for Anthropic's Claude on AWS Bedrock

Behavioral testing for LLM applications. pytest plugin with semantic assertions, multi-turn conversation testing, and drift detection. No LLM judge needed.

CLI tool for comparing LLM API pricing, ranked by cost-effectiveness against LMSYS Arena scores

Benchmark any LLM provider against your actual prompts — latency, cost, quality

Unified Python library for LLM APIs (OpenAI, Anthropic, Gemini, xAI, Groq, custom)

LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the diverse strengths and weaknesses of multiple open-source large language models (LLMs). LLM-Blender cut the weaknesses through ranking and integrate the strengths through fusing generation to enhance the capability of LLMs.

Pre-flight LLM cost estimation and budget enforcement

Prevent LLM cache poisoning with Confidence Gap Analysis — intent-aware thresholds, zero false serves

Agent Cache Runtime for building cache-aware LLM applications.

Semantic cache, multi-provider LLM router and cost tracker (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen)

Lightweight caching layer for LLM API responses with TTL and size limits

LLM plugin for Anthropic's Claude

llm-claude-3 is now llm-anthropic

Production-grade LLMOps infrastructure for context window management, token counting, document chunking, and compression

Composable building blocks for LLM context engineering

Cost tracking for OpenAI, Gemini, and Claude APIs with session management

Real-time cost monitoring and budget enforcement for LLM API calls

Python SDK for LLM Cost Monitor - Track, aggregate, and analyze LLM usage costs

Automatically reduce LLM API costs. Routes to cheapest model that succeeds. Use kalibr instead.

Reduce LLM costs automatically with outcome-based routing. Use kalibr instead.

Track LLM API costs per request. Know where your tokens go.

A minimal, auditable Python library for calling multiple LLM providers. Lightweight LiteLLM alternative.

LLM model discovery and tracking system for real-time monitoring of available models across multiple providers

Carbon-aware scheduling and telemetry for LLM applications

Create ensembles of Large Language Models to query them all at the same time.

CLI tool to expose LLMs through messaging clients (Telegram, Discord, Slack, etc.)

Multi-LLM provider client with automatic failover and priority ordering

Automatic failover between LLM providers. When OpenAI is down, seamlessly switch to Anthropic, Google, or any backup.

LLM plugin for analyzing FEC campaign finance filings

A thin, fast, observable Python client for LLMs

AI-powered filesystem cleanup tool with interactive CLI

Run URLs through the pure.md API

Run URLs through the Jina Reader API

A plugin for llm to load Reddit threads as fragments

Run URLs through the Trafilatura to get plain text

A unified handler for LLM API calls across OpenAI, Anthropic, and Google

CLI-based threat hunting platform using LLM and RAG for semantic search of security logs

Drop-in prompt injection defense for LLM apps and AI agents — detect, sanitize, block, and audit injection attacks in real time. Includes multi-turn session scanning, allow-lists, rate-abuse detection, multi-layer scanner, FastAPI and Flask middleware.

A Python library for managing multi-agent model invocation with automatic failover strategies

A unified interface for streaming structured JSON from OpenAI, Anthropic, and Google Gemini.

One tiny model, every LLM API. Drop-in test server for OpenAI, Anthropic, Bedrock, and Vertex.

A unified toolkit for working with multiple LLM providers

Cost ceiling, audit log, and kill switch for LLM agents.

A thread-safe and async rate limiter for Gemini and OpenAI models.

Lightweight LLM call logger for OpenAI/Anthropic

Stable canonical sha256 hash of LLM request/message structures. Recursive key-sorted JSON canonicalization with per-provider presets that drop noise fields. For cache keys and idempotency. Zero runtime deps.

pytest plugin to mock and replay OpenAI and Anthropic API calls — record once, replay forever, no API key needed

Track and check deprecation status of LLM provider models (OpenAI, Anthropic, etc.)

Compare LLM model outputs side-by-side with rich diff visualization

Intelligent multi-LLM request router for cost optimization

Mutation testing for LLM prompts. Find the gaps in your eval suite before production does.

Lightweight SDK for LLM inference logging and observability

Production-ready LLM platform building blocks — observability, prompts, eval, RAG, hallucination guards

Local preflight checks for LLM provider keys, endpoints, and model configuration

Reliable, structured, production-safe LLM outputs with schema validation and auto-repair

A unified interface for multiple LLM providers

LLM Proxy Server is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

Real-time observability dashboard for LLM applications. Track prompts, tokens, costs, and latency. One-line integration.

MCP server providing real-time AI model intelligence - pricing, capabilities, and recommendations

Cloud-agnostic rate limit mitigation for LLM APIs

A centralized registry for discovering and managing LLM model capabilities. Track model features, costs, and limitations across providers like OpenAI and Anthropic. Supports both verified model definitions and user-managed entries with local storage.

SQLite-backed LLM response cache. Exact match + fuzzy match. Decorator API. Zero mandatory server dependencies.

LLM-Rosetta: Unified LLM API adapter — bidirectional conversion between OpenAI, Anthropic, and Google formats via intermediate representation (IR).

Fault-tolerant LLM provider rotation with circuit breaker, quotas, and model-first routing

Intelligent LLM routing with weighted least-outstanding selection, 429 cooldown, and request class isolation

Route text and embedding calls across LLM providers, keeping a JSONL ledger of every request and response for offline cost reconciliation.

DEPRECATED — use pii-veil + pii-core instead. Reversible PII anonymization for Polish documents.

A Python library for selecting the best LLM model based on user input using any LLM via LiteLLM

Multi-provider AI chat interface supporting 9+ LLM providers (Anthropic, OpenAI, xAI, Cohere, Mistral, Perplexity, Gemini, HuggingFace, Groq)

輕量級 LLM 模型追蹤與自動替換 Decorator

Track and limit LLM API spending in real-time. Drop-in middleware for OpenAI, Anthropic, Google, and any OpenAI-compatible API.

A comprehensive LLM tracing and monitoring library

Agent framework with tool calling for OpenAI, Anthropic, and Google

Python library for standardization, optimization and monitoring of LLM API calls