Agent frameworks & infrastructure
Observability & evaluation
5,667 agents · most starred first
Observability & evaluation, page 26 of 57
streamlit region:us
streamlit region:us
docker region:us
streamlit region:us
gradio region:us
static region:us
Find out whether your AI agent's evals are lying to you: stability reports with flip attribution, LLM judge audits, and deterministic checks mined from your KB.
gradio region:us
Multi-agent observability: cascade failure detection, heartbeat monitoring, cross-agent tracing, and forensic replay
Catch silent failures in agent workflows by verifying real system state
Future AGI review: compare features, pricing, use cases, access model, and alternatives for this Observability agent in 2026. Most accurate evaluation agents that work across all modalities
Maxim AI review: compare features, pricing, use cases, access model, and alternatives for this Observability agent in 2026. Rated 5/5 from 1 verified review. Evaluate and improve AI Agents, faster
Confident AI review: compare features, pricing, use cases, access model, and alternatives for this Observability agent in 2026. The DeepEval LLM Evaluation Platform
Arize AI review: compare features, pricing, use cases, access model, and alternatives for this Observability agent in 2026. AI observability and LLM evaluation platform for monitoring and improving ML models
Athina helps your team build, test, and monitor AI features faster.
Oversee AI agents, ensuring they perform well and stay compliant.
Monitor and debug your AI applications to ship reliable features faster.
Agenta helps teams build, test, and manage reliable AI applications.
Automate AI testing and ensure your AI apps always work well.
Understand, debug, and improve your AI applications easily.
Detect if code is running in an AI agent or automated development environment
n8n community node: AI Agent + Langfuse
Real-time cost enforcement for AI agent sessions — works with OpenAI and Anthropic
Arena CLI - AI Agent Competition Platform
Wayfound AI Agent management platform
Admin monitoring and control dashboard for Gunma AI Agent
Transparent AI agent tracing
Measures how well an AI agent can navigate a codebase autonomously without human assistance
Monitor and control AI agent sessions from any browser.
A real-time dashboard for monitoring AI agent sessions (Claude Code, Codex) with 3D visualization
TypeScript type definitions and semantic convention constants for [AgentPrism](https://github.com/evilmartians/agent-prism) - AI agent trace visualization library.
AI agent observability and control from your iPhone or Apple Watch
Read-only peek into other AI agent chat sessions.
Real-time graph visualizer for AI agent execution flows
AI agent evaluation CLI for races, replay, scorecards, and CI regression gates
Local-first AI agent observability SDK (TypeScript)
Executable AI agent governance with workflow gates, evidence, skill/tool orchestration, and traceable HTML artifacts
AI Agent Management Dashboard — monitor and manage Claude Code, GitHub Copilot, and other AI agent sessions
Assertion framework for AI agent evaluations - supports skill invocation checks, build validation, and LLM-based judging
Data transformation utilities for [AgentPrism](https://github.com/evilmartians/agent-prism) - converting OpenTelemetry and Langfuse traces to UI-ready formats for AI agent trace visualization.
Locale-aware eval harness for AI agent behavior
Panopticon Command Deck — AI agent monitoring desktop app
LLM Agent Framework and Observe connector
OpenTelemetry GenAI utility for standardized telemetry collection across LLM, Agent, Embedding, Tool, Retrieval, Rerank, Memory and more
AI-Native Logging for LLM Agent Development - TypeScript/Node.js Implementation
A framework for tracing, tracking and evaluating LLM agent pipelines
⚡️ AI 系统测试框架 — 面向 LLM/Agent 的轻量断言测试工具,AI 界的 Vitest
Framework-agnostic LLM agent evaluation harness
Datadog LLM Observability exporter for Mastra - exports tracing data to Datadog's LLM Observability product
LLM agent evaluation framework — define agent behaviour in YAML, evaluate against multiple models, compare results
Enforce real-time token budgets and spending limits for OpenAI, Anthropic Claude, and Google Gemini API calls in Node.js
Lightweight, zero-dependency LLM API cost & token usage tracker for OpenAI, Anthropic, Gemini, Mistral, Groq, and DeepSeek
Canvas-rendered React component for real-time LLM agent telemetry — live token rates, progress, status transitions, and health scoring.
TES SDK — LLM observability and lifecycle tracking via Pentatonic Thing Event System. Track token usage, tool calls, and conversations. Manage things through event-sourced lifecycle stages with AI enrichment and vector search.
Middleware-powered eval framework for Claude Code. Intercepts the Anthropic API to observe and scrub tool calls.
Intelligent LLM agent cost optimization runtime. Coming soon.
Guardrail regression testing for LLM agent tool calls
Client library to connect to the LangSmith Observability and Evaluation Platform.
Cost-tracking middleware for @diabolicallabs/llm-client. Async fire-and-forget ingestion to Agent Spend Dashboard. © Diabolical Labs
Amplitude AI SDK - LLM usage tracking for Amplitude Analytics
Run LangWatch locally with one command — `npx @langwatch/server`. Complete LangWatch stack: observability, evaluations, AI gateway, agent simulations.
Agent Evaluation and Observability Framework
Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP
PostHog LLM Analytics extension for pi coding agent
A module for generating metrics from V8.
CLI to evaluate agent skills triggering and functionality
New Relic agent
Simple OpenAI compatible Mock API server. Useful for deterministic testing of AI applications.
LLM eval & testing toolkit
Embedded agent simulation & evaluation framework for Node.js/TypeScript
Elastic APM JavaScript agent
Package to connect and trace LLM calls.
Raindrop observability for Pi Agent — automatic tracing via subscriber or pi-coding-agent extension
Agent evaluation framework — prove whether agent changes improved outcomes with reproducible evidence.
Fake LLM adapter for Flink AI framework - perfect for demos and testing
New Relic Browser Agent
ReportPortal client for Node.js
[](https://dl.circleci.com/status-badge/redirect/gh/taneliang/coerce-llm-output/tree/main) [
The official Elastic APM agent for Node.js
Traceloop Software Development Kit (SDK) for Node.js
OpenTelemetry instrumentation for LangchainJS
Langfuse integration for LangChain
Core functions and utilities for Langfuse packages
Langfuse OpenTelemetry export helpers
Langfuse instrumentation methods based on OpenTelemetry
Langfuse API client for universal JavaScript environments
[Agentic applications](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/) give an LLM freedom over control flow in order to solve problems. While this freedom can be extremely powerful, the black box nature of LLMs can make it difficult
LEGACY Langfuse v3 integration for LangChain (maintenance mode). For new projects use the current CallbackHandler from @langfuse/langchain: npm install @langfuse/langchain @langfuse/otel
SDK for integrating Braintrust with LangChain.js
OpenInference Instrumentation for LangChain.js
Langfuse integration for OpenAI SDK
Verification, observability, and quality scoring utilities for LangChain tool-calling agents. Includes confidence scoring, data freshness validation, numerical consistency checks, ticker symbol hallucination detection, error categorization, token tracking
Microsoft Agent 365 SDK LangChain instrumentation extensions for telemetry and observability
Opik TypeScript and JavaScript SDK integration with LangChain
LangChain tools and observability integrations for agent-context-kit
OpenInference-compatible observability for AG-Kit