Agent frameworks & infrastructure
Observability & evaluation
5,667 agents · most starred first
Observability & evaluation, page 27 of 57
LaunchDarkly AI SDK LangChain Provider for Server-Side JavaScript
Framework integrations for Agent Replay
OpenInference instrumentation for Claude Agent SDK
XAIP callback handler for LangChain.js — emits signed XAIP receipts for every tool call. Provider-agnostic trust layer, drop-in.
This is the langchain wrapper built for (Maxim JS SDK)[https://www.npmjs.com/package/@maximai/maxim-js].
Proviras tracing SDK for LangGraph and LangChain agents
Elastic apm core
Raindrop integration for LangChain
OpenTelemetry GenAI tracing for LangChain/LangGraph JS with Azure AI Foundry and Application Insights
Guards, Evals & Observability for AI applications - works seamlessly with LangChain/LangGraph
Capture the complete train of thought of agent work as first-class artifacts
Elastic APM Real User Monitoring for React applications
NotiLens callback handler for LangChain.js — automatic run tracking, token metrics, and error alerting
Evaluation framework with Effect-based architecture for running parallel AI evaluations
OpenCode plugin for tracing to LangSmith
Official JavaScript/TypeScript SDK for the Execlave AI agent governance platform — tracing, policy enforcement, and framework auto-instrumentation for LangChain, OpenAI Agents SDK, CrewAI, Model Context Protocol, and OpenAI Chat Completions
JS client for Galileo
Struct agent observability SDK — auto-instruments AI agent frameworks with OpenTelemetry
LangChain library that logs model inputs/outputs with customizable formatting
LangGraph.js / LangChain.js callback handler that emits Goderash audit events — tamper-evident, hash-chained, regulator-ready.
Lunary is an open-source platform for AI developers.
A New Relic Mobile Agent for React Native
A dashboard for the Convex Agent component.
OpenTelemetry instrumentation for the `openai` OpenAI client library
SDK for integrating Braintrust with OpenAI Agents
OpenAI Instrumentaion
Agent Runtime Control — Loop detection, prompt caching, budget enforcement, and waste scanning for autonomous AI agents.
Microsoft Agent 365 SDK OpenAI Agents instrumentation extensions for telemetry and observability
n8n community node: Langfuse + OpenAI-compatible LLM provider
Azure OpenAI Instrumentaion
Together Instrumentation
Azure Monitor Application Insights observability adapter for @cuylabs/agent-core
Raindrop integration for Mastra
OpenTelemetry ai-specific semantic conventions
Behavior contracts for AI agents — tested in your PR, enforced at runtime. (TypeScript SDK)
Spanlens SDK — agent tracing, LLM usage capture, and cost observability for TypeScript.
See exactly what your coding agent sends to the model — a local logging reverse-proxy + web dashboard for Claude Code, Codex, DeepSeek-TUI, Reasonix, and Kimi. Just run `ccglass`.
Opik TypeScript and JavaScript SDK integration with OpenAI
Estimate the number of tokens an OpenAI chat completion request will use
AnoSys SDK for OpenAI Agents — automatic instrumentation and logging for OpenAI Agents SDK
Raindrop integration for OpenAI Agents SDK
Voight observability for the OpenAI SDK. Wrap your OpenAI client and capture every model call — prompts, tokens, costs, latency, errors — surfaced live in the Voight dashboard.
OpenInference instrumentation for OpenAI
PostHog Node.js AI integrations
Armalo AI integrations for OpenAI, Anthropic, Gemini, LangGraph, CrewAI — 2-line trust telemetry.
LangSmith-native autonomous agent optimization for Claude Code
Local-first deterministic CLI that scores whether an issue, spec, or decision artifact is ready for autonomous agent work. No LLM calls, no hosted backend.
Evaluation library for the MongoDB Assistant API.
A local development tool for debugging and inspecting AI SDK applications. View LLM requests, responses, tool calls, and multi-step interactions in a web-based UI.
A lightweight registry of LLM model information, like name and context sizes, for building AI-powered apps.
Recall Desktop SDK
AI development tools for debugging and monitoring AI SDKs streams
Microsoft Application Insights JavaScript SDK - Web
OpenInference utilities for ingesting Vercel AI SDK spans
SDK for integrating Braintrust
LaunchDarkly AI SDK for Server-Side JavaScript
Core TanStack AI Devtools plugin for inspecting chat messages, tool calls, streams, and errors.
Opik TypeScript and JavaScript SDK integration with Vercel AI SDK
OpenTelemetry instrumentation for `aws-sdk` and `@aws-sdk/client-*` clients for various AWS services
Standalone Vercel AI SDK integration for Raindrop (events + OTLP/HTTP JSON traces, no OTEL runtime)
React Devtools plugin for inspecting TanStack AI chat messages, tool calls, streams, and errors.
OpenInference instrumentation for Anthropic
Structured diagnostic code library
Axiom AI SDK provides - an API to wrap your AI calls with observability instrumentation. - offline evals - online evals
Helicone provider for Vercel AI SDK
Vercel AI SDK integration for MLflow Tracing — SpanProcessor that translates AI SDK span attributes to MLflow format
Microsoft Application Insights JavaScript SDK - 1ds-core-js
Cost calculator for Vercel AI SDK token usage. Supports OpenAI, Anthropic, Google, xAI (Grok), and DeepSeek with long context pricing, prompt caching, and reasoning tokens.
Google Gen AI SDK Instrumentation
Primary Raindrop JavaScript SDK for event tracking, AI event ingestion, self-diagnostics, and tracing helpers.
Vibe Coding Tracker - AI coding assistant telemetry/usage parser, aggregate JSONL events into CodeAnalysis results
Solid Devtools plugin for inspecting TanStack AI chat messages, tool calls, streams, and errors.
The official Fullstory browser SDK
> ⚠️ **Beta Notice** > > This SDK is currently in **beta**. All APIs are **experimental** and subject to change. > Please review the [release notes](https://github.com/statsig-io/statsig-ai-node/releases) for any **breaking changes** before upgrading.
Real-time visualizer for Claude Code and GitHub Copilot CLI sessions: live event stream, token tracking, multi-agent topology with per-source badges, plus a token observatory built on the bundled netgain analysis engine.
@multi-agent/logger
Ambient multi-agent observability for Pi — a terminal habitat for coding agents.
Evaluation suite for swarm-tools multi-agent coordination
Multi-agent observability: cascade failure detection, heartbeats, and forensic replay
PRISM — Agentic visibility layer for Claude. Intercepts, records, and replays multi-agent sessions.
Cross-framework debugger SDK for multi-agent AI systems
CLI tool to monitor agent status in real-time. Works with Claude Code, multi-agent workflows, and any AI agent system.
Ed25519-signed, SHA-256-chained tool-call audit log for OpenClaw multi-agent setups. Tamper-evident provenance chain.
The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.
A framework for capturing, benchmarking, and visualizing agent goals, implementations, and runs.
Maiar: A Composable, Plugin-Based AI Agent Framework
Agents Evaluation API for Inkeep Agent Framework - handles evaluations, datasets, and evaluation runs
Adapters that connect any AI agent framework (JS/TS) to the P2PCLAW BenchClaw benchmark leaderboard.
Lightweight SDK for agent observability. Install in your agent framework to capture events.
Core utilities for agent-tail log capture plugins.
Evaluation framework for Axl agentic workflows
Agentic AI Library specialized in LLM Function Calling
Benchmark agentic coding AIs against coding tasks
A visualizer for Railtracks agentic flows
AI agent debugging, collaboration, and trace observability. Built for teams using CrewAI, OpenAI, and more.
Zero-code runtime observability for JS/Python + AI agent debugging. Traces LangChain, CrewAI, OpenAI, Anthropic, Gemini. Eval, security, compliance, cost tracking. Free, local-first.
Open-source testing and regression detection framework for AI agents. Golden baseline diffing, CI/CD integration, works with LangGraph, CrewAI, OpenAI, Claude, HuggingFace, Ollama, and MCP.
Framework-agnostic agent observability & LLM tracing. Supports OpenAI, CrewAI, AutoGen, LangGraph, and OpenTelemetry.
OpenTelemetry SpanProcessor for atrib's verifiable action layer. Emits signed records from OpenInference-shaped spans.
Monitor any AI agent system. Auto-detects failures, sends alerts, audits OS processes. Zero config, zero dependencies.