Python agents

A developer-centric CLI tool to systematically evaluate and compare Large Language Models (LLMs)

CLI tool for comparing LLM API pricing, ranked by cost-effectiveness against LMSYS Arena scores

LLM evaluation harness with custom metrics, LLM-as-judge, and regression tracking

LLM Benchmark

MCP Server for LLM comparison, benchmarks, and pricing — find the best model for any task

LLM Inference Benchmark CLI - measure TTFT, TPS, ITL, E2E latency for any OpenAI-compatible API

Benchmark LLMs with 10 benchmarks & 132K+ questions. 8 providers: OpenAI, Anthropic, Groq, Together, Fireworks, DeepSeek, Ollama, HuggingFace. Unified CLI + Web dashboard.

Benchmark any LLM provider against your actual prompts — latency, cost, quality

LLM trainer

Unified Python library for LLM APIs (OpenAI, Anthropic, Gemini, xAI, Groq, custom)

LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the diverse strengths and weaknesses of multiple open-source large language models (LLMs). LLM-Blender cut the weaknesses through ranking and integrate the strengths through fusing generation to enhance the capability of LLMs.

Simple interface for creating and managing LLM chains

Python library for developing LLM bots

Pre-flight LLM cost estimation and budget enforcement

Prevent LLM cache poisoning with Confidence Gap Analysis — intent-aware thresholds, zero false serves

Lite library for local caching of LLM API requests

Agent Cache Runtime for building cache-aware LLM applications.

Check whether you've already run this prompt before calling the LLM

A drop-in, model-agnostic cache for Large Language Model API calls

Local SQLite cache for OpenAI and Anthropic API calls. One env var, 60-80% cheaper dev loops.

Semantic cache, multi-provider LLM router and cost tracker (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen)

Lightweight caching layer for LLM API responses with TTL and size limits

Cache LLM API calls to speed up development and prevent rate limits

Unified Language Model Cache.

LLM inference hardware calculator — architecture-aware, engine-version-aware, honest-labeled.

Simple Azure OpenAI call helper

Visualize complex LLM conversation flows in infinite canvases

Calculate CO2 emissions from LLM API calls following Green Software Foundation standards.

LLM plugin for automatically mapping and describing a codebase or project

llm api and prompt

A Python library that uses LLMs to diagnose and explain exceptions

Plugin for LLM adding fast Cerebras inference API support

llm-change-agent

A CLI tool for chatting with multiple LLM providers

A short description of the project

chat api collection for popular LLM

Chat with LLMs from the terminal

Developer-first kit for LLM chatbots with streaming, memory, and personas (Discord-first)

A universal terminal chat client that auto-detects and connects to any chat API endpoint

A series of coherence and calibration checks for LLMs.

CLI tool to check hardware for local LLMs

common functions, utilities using multiple DB and LLM options.

Semantic-aware chunking with provenance tracking for production RAG and LLM data pipelines

Add your description here

LLM plugin for models hosted by Chutes AI

A simple CI/CD utility for running LLM tasks with Semantic Kernel

LLM plugin for verifying academic citations against Crossref

Generate embeddings for audio files using CLAP with llm

Structured LLM based classification, clustering and extraction framework that works with all major API providers

Framework to build production-grade LLM-based classifiers

LLM plugin for content classification using various language models

LLM plugin for Anthropic's Claude

llm-claude-3 is now llm-anthropic

LLM plugin for Claude models via Claude Code CLI

LLM plugin for Claude Code SDK integration

LLM plugin for Claude Code integration

A robust JSON repair and extraction tool for LLM outputs.

llm plugin for multi-LLM orchestration via llm-cli-gateway

A lightweight command-line tool that generates terminal commands from natural language

Universal AI CLI Proxy, Firewall, and Token Intelligence Gateway

A CLI tool to interact with various LLM APIs

SDK for using LLM

LLM redis中介的客户端程序

A reusable library for LLM clients with LangChain framework

Generate embeddings for images and text using CLIP with LLM

An LLM CLI plugin for Cloudflare Workers AI models.

LLM plugin for clustering embeddings

A drop-in replacement for sklearn clustering models, this package optimizes sklearn clusters using LLMs

Use LLM to generate and execute commands in your shell

Use LLM to generate commands for your shell

Use LLM to generate commands for your shell: specialized for qwen3

Python CLI tool that leverages LLMs to generate and execute shell commands through natural language input

Multi-user web cockpit for Ollama: dashboard + chat + admin, pip-installable in one process

An OpenAI LLM based CLI coding assistant.

A tool to help with copying and pasting context into LLM chats (Deprecated: Renamed to llm-context)

A tool to generate comprehensive context for LLMs from any codebase.

Placeholder package for llm-dev

python code executor tool that takes fuzzy input text to extract and execute code designed for LLM applications

Ultra-fast intelligent code analysis tool for LLM context generation with Tree-sitter

Command line tool to code review GitHub pull requests with LLMs.

CLI guardrail for catching stale Python APIs before runtime.

Useful classes and methods for researching code-generation by LLMs.

Generate structured Context Packs for AI coding tools by analyzing your codebase.

llm による自立型 Cli コーディングエージェントライブラリ

Plugin for LLM adding support for Cohere's Generate and Summarize models

Access the Cohere Command R family of models

A plugin for llm that generates commit messages

LLM-friendly replacement for git diff --staged

A Python wrapper for managing OpenAI API and other LLM models.

Dynamic serializer for LLM Compact Hierarchical Formats.

LLM Comparator: An interactive visualization tool for side-by-side LLM evaluation

LLM plugin to compare responses from multiple language models

A JupyterLab extension.

Use LLM to generate commands for your shell

A library for LLM text completion using LiteLLM

Llm Compliance Comparison tools for AI agents. Capabilities: compare providers, recommend for use case, provider risk profile. Built by MEOK AI Labs.

A library of ready-to-use LLM components

Arithmetic compression using Llama LLM models

A provider-agnostic SDK for LLM, image, video, and embedding model clients