PyPI agents

Production-ready LLM client with audit trail, deterministic caching, and provenance tracking

llm-lens is a Python package for CV as NLP, where you can run very descriptive image modules on images, and then pass those descriptions to a Large Language Model (LLM) to reason about those images.

Automatic observability for LLM API calls

Local web UI for browsing and managing LLM CLI conversation history (currently: Claude Code).

A thread-safe and async rate limiter for Gemini and OpenAI models.

LLM-oriented CLI linter — catch quality drift before it hardens (Rust binary).

A Python package to list and monitor available LLM models from various providers

LLM plugin for LiteLLM proxy server

LLM plugin for LiteLLM OpenAI-compatible proxies

Use LLM to chat in your terminal and receive async responses

LLM plugin for running models using llama.cpp

Interact with llama-server models

LLM plugin providing access to llama.cpp server models

Access llamafile localhost models via LLM

A middleware layer for managing, budgeting, and optimizing LLM context windows.

LLM plugin for models served by LMStudio (local LLM studio HTTP API)

Agentic interface to local LLMs with tree-based context management.

Structured output contract testing for LLM systems

AI-powered log analyzer for CI environments using LLMs

Lightweight LLM call logger for OpenAI/Anthropic

Set logging.DEBUG while running LLM

Privacy-first LLM chat log parser for deterministic local analysis and export

Feedback on LLM logs (prompts & response)

Give your AI agents persistent, searchable long-term memory with pluggable storage backends

A utility package for querying language models with pattern matching and retry logic

An LLM CLI plugin to loop tool calls with turn limits and custom system prompts.

Lucid Memory - Modular reasoning graph for LLMs

multi-agent development

LLM Magics for Ipython created by kmkolasinski

the small distributed language model toolkit. fine-tune state-of-the-art LLMs anywhere, rapidly.

Flash-built LLM workflows

MapReduce-inspired framework for extending context windows in large language models

Turn Python functions into typed LLM calls using docstrings as prompts

Convert PDFs, images to high-quality Markdown using Vision LLMs.

Plugin for LLM adding a Markov chain generating model

Mask sensitive data in documents using a local OpenAI-compatible LLM

An llm plugin that provides a matched-responses model for testing - returns configurable responses based on input patterns

Retrieval-backed LLMs for math education

llm_matrix

llm implementation of MCP for clients and servers.

LLM plugin for Model Context Protocol (MCP) integration

A minimal MCP service for testing LLM understanding and invocation of MCP tools

An experimental project using Monte Carlo Tree Search (MCTS) to refine LLM responses for better accuracy and decision-making.

llm-mddocgen - Large Language Model Markdown Document Generator

Guardrails for LLM-generated medical and health content

lllm-mem is a lightweight Python CLI tool designed to calculate the GPU VRAM requirements for models on Hugging Face

A LLM Memory Cache package

Estimate LLM VRAM usage (weights/KV/activations) and recommend tp/dp/sp configs.

Personal conversation knowledge base: import, search, and analyze conversations from ChatGPT, Claude, Gemini, and Claude Code

A memory management system for large language models

Currently for internal use only.

Stable canonical sha256 hash of LLM request/message structures. Recursive key-sorted JSON canonicalization with per-provider presets that drop noise fields. For cache keys and idempotency. Zero runtime deps.

A helper library for estimating tokens used by messages.

A tool for harvesting metadata from dataset landing pages using Large Language Models.

Accurate LLM usage & cost tracking for Python backends (FastAPI-native)

A lightweight evaluation framework for large language model outputs

Official implementation of the LLM-Microscope package functions for the paper

LLM MIME types

Generates LLM context by scraping and summarizing documentation for Python libraries listed in a requirements.txt file.

A framework to glue all your AI tools together

LLM plugin for accessing MiniMax models via their API

LLM plugin providing access to Mistral models using the Mistral API

LLM plugin for running models using MLC

Support for MLX models in LLM

pytest plugin to mock and replay OpenAI and Anthropic API calls — record once, replay forever, no API key needed

Mock LLMs' responses for unit-tests

Standalone model alias resolver

A package to calculate token costs for LLM models

Track and check deprecation status of LLM provider models (OpenAI, Anthropic, etc.)

Compare LLM model outputs side-by-side with rich diff visualization

Filter llm models list by starring favorites

OpenAI-compatible API server for simonw's llm cli

Intelligent multi-LLM request router for cost optimization

A CLI tool to list available LLM models from various providers

📈 Monitor your LLM integration with Galileo's LLM Monitor!

Lightweight SDK for LLM cost & latency monitoring

A small example package

Run prompts against LLMs hosted by Moonshot

Plugin for LLM adding support for the MPT-30B language model

Map-reduce spreadsheet operations powered by LLM prompts

Chat with Mr Chatterbox

A plugin for LLM that combines file bundling and LLM mapping capabilities

A Python library for using Large Language Models (LLMs) to fill out multiple-choice questionnaires about chat conversations.

LLM API for multiple model providers

Mutation testing for LLM prompts. Find the gaps in your eval suite before production does.

Deterministic VM for LLM program execution

Advanced system information tool for local LLM usage

Schema-driven Named Entity Recognition powered by local LLMs via Ollama

Use an LLM to run inference, for NER (named entity recognition) and/or NEL (named entity linking). Build a knowledge graph. Evaluate results.

A plugin to add support for the Neuralwatt AI inference service to llm

A unified interface for multiple language model providers

Create embeddings using the Nomic API

LLM plugin for models hosted by NOS Cloud

Karpathy-style LLM wiki from your Claude Code, Codex CLI, Cursor, and Obsidian sessions

Access Nous Research models via API

A tool for obfuscating text by manipulating token IDs while preserving token count and structure

Lightweight SDK for LLM inference logging and observability

Context0 SDK — one import, all calls logged

OpenAI-compatible LLM proxy with SQLite request capture, observability, and an admin UI.