Agent frameworks & infrastructure

Observability & evaluation

5,667 agents · most starred first

Observability & evaluation, page 27 of 57

@launchdarkly/server-sdk-ai-langchain

LaunchDarkly AI SDK LangChain Provider for Server-Side JavaScript

@reaatech/agent-replay-integrations

Framework integrations for Agent Replay

@arizeai/openinference-instrumentation-claude-agent-sdk

OpenInference instrumentation for Claude Agent SDK

xaip-langchain

XAIP callback handler for LangChain.js — emits signed XAIP receipts for every tool call. Provider-agnostic trust layer, drop-in.

@maximai/maxim-js-langchain

This is the langchain wrapper built for (Maxim JS SDK)[https://www.npmjs.com/package/@maximai/maxim-js].

proviras-sdk

Proviras tracing SDK for LangGraph and LangChain agents

@elastic/apm-rum-core

Elastic apm core

@raindrop-ai/langchain

Raindrop integration for LangChain

@mndr/langchain-azure-otel

OpenTelemetry GenAI tracing for LangChain/LangGraph JS with Azure AI Foundry and Application Insights

@dooor-ai/toolkit

Guards, Evals & Observability for AI applications - works seamlessly with LangChain/LangGraph

agent-trajectories

Capture the complete train of thought of agent work as first-class artifacts

@elastic/apm-rum-react

Elastic APM Real User Monitoring for React applications

@notilens/langchain

NotiLens callback handler for LangChain.js — automatic run tracking, token metrics, and error alerting

@orq-ai/evaluatorq

Evaluation framework with Effect-based architecture for running parallel AI evaluations

@langchain/langsmith-opencode

OpenCode plugin for tracing to LangSmith

@execlave/sdk

Official JavaScript/TypeScript SDK for the Execlave AI agent governance platform — tracing, policy enforcement, and framework auto-instrumentation for LangChain, OpenAI Agents SDK, CrewAI, Model Context Protocol, and OpenAI Chat Completions

galileo

JS client for Galileo

@struct-ai/sdk

Struct agent observability SDK — auto-instruments AI agent frameworks with OpenTelemetry

langchain-logging

LangChain library that logs model inputs/outputs with customizable formatting

@goderash/adapter-langgraph

LangGraph.js / LangChain.js callback handler that emits Goderash audit events — tamper-evident, hash-chained, regulator-ready.

lunary

Lunary is an open-source platform for AI developers.

newrelic-react-native-agent

A New Relic Mobile Agent for React Native

@convex-dev/agent-playground

A dashboard for the Convex Agent component.

@opentelemetry/instrumentation-openai

OpenTelemetry instrumentation for the `openai` OpenAI client library

@braintrust/openai-agents

SDK for integrating Braintrust with OpenAI Agents

@traceloop/instrumentation-openai

OpenAI Instrumentaion

vibe-billing

Agent Runtime Control — Loop detection, prompt caching, budget enforcement, and waste scanning for autonomous AI agents.

@microsoft/agents-a365-observability-extensions-openai

Microsoft Agent 365 SDK OpenAI Agents instrumentation extensions for telemetry and observability

n8n-nodes-openai-langfuse

n8n community node: Langfuse + OpenAI-compatible LLM provider

@traceloop/instrumentation-azure

Azure OpenAI Instrumentaion

@traceloop/instrumentation-together

Together Instrumentation

@cuylabs/agent-azure-monitor

Azure Monitor Application Insights observability adapter for @cuylabs/agent-core

@raindrop-ai/mastra

Raindrop integration for Mastra

@traceloop/ai-semantic-conventions

OpenTelemetry ai-specific semantic conventions

shadow-diff

Behavior contracts for AI agents — tested in your PR, enforced at runtime. (TypeScript SDK)

@spanlens/sdk

Spanlens SDK — agent tracing, LLM usage capture, and cost observability for TypeScript.

ccglass

See exactly what your coding agent sends to the model — a local logging reverse-proxy + web dashboard for Claude Code, Codex, DeepSeek-TUI, Reasonix, and Kimi. Just run `ccglass`.

opik-openai

Opik TypeScript and JavaScript SDK integration with OpenAI

openai-chat-tokens

Estimate the number of tokens an OpenAI chat completion request will use

anosys-sdk-openai-agents

AnoSys SDK for OpenAI Agents — automatic instrumentation and logging for OpenAI Agents SDK

@raindrop-ai/openai-agents

Raindrop integration for OpenAI Agents SDK

@voightxyz/openai

Voight observability for the OpenAI SDK. Wrap your OpenAI client and capture every model call — prompts, tokens, costs, latency, errors — surfaced live in the Voight dashboard.

@arizeai/openinference-instrumentation-openai

OpenInference instrumentation for OpenAI

@posthog/ai

PostHog Node.js AI integrations

@armalo/integrations

Armalo AI integrations for OpenAI, Anthropic, Gemini, LangGraph, CrewAI — 2-line trust telemetry.

harness-evolver

LangSmith-native autonomous agent optimization for Claude Code

@jawnty/agent-preflight

Local-first deterministic CLI that scores whether an issue, spec, or decision artifact is ready for autonomous agent work. No LLM calls, no hosted backend.

mongodb-assistant-eval

Evaluation library for the MongoDB Assistant API.

@ai-sdk/devtools

A local development tool for debugging and inspecting AI SDK applications. View LLM requests, responses, tool calls, and multi-step interactions in a web-based UI.

tokenlens

A lightweight registry of LLM model information, like name and context sizes, for building AI-powered apps.

@recallai/desktop-sdk

Recall Desktop SDK

@ai-sdk-tools/devtools

AI development tools for debugging and monitoring AI SDKs streams

@microsoft/applicationinsights-web

Microsoft Application Insights JavaScript SDK - Web

@arizeai/openinference-vercel

OpenInference utilities for ingesting Vercel AI SDK spans

braintrust

SDK for integrating Braintrust

@launchdarkly/server-sdk-ai

LaunchDarkly AI SDK for Server-Side JavaScript

@tanstack/ai-devtools-core

Core TanStack AI Devtools plugin for inspecting chat messages, tool calls, streams, and errors.

opik-vercel

Opik TypeScript and JavaScript SDK integration with Vercel AI SDK

@opentelemetry/instrumentation-aws-sdk

OpenTelemetry instrumentation for `aws-sdk` and `@aws-sdk/client-*` clients for various AWS services

@raindrop-ai/ai-sdk

Standalone Vercel AI SDK integration for Raindrop (events + OTLP/HTTP JSON traces, no OTEL runtime)

@tanstack/react-ai-devtools

React Devtools plugin for inspecting TanStack AI chat messages, tool calls, streams, and errors.

@arizeai/openinference-instrumentation-anthropic

OpenInference instrumentation for Anthropic

logs-sdk

Structured diagnostic code library

axiom

Axiom AI SDK provides - an API to wrap your AI calls with observability instrumentation. - offline evals - online evals

@helicone/ai-sdk-provider

Helicone provider for Vercel AI SDK

@mlflow/vercel

Vercel AI SDK integration for MLflow Tracing — SpanProcessor that translates AI SDK span attributes to MLflow format

@microsoft/1ds-core-js

Microsoft Application Insights JavaScript SDK - 1ds-core-js

ai-sdk-cost-calculator

Cost calculator for Vercel AI SDK token usage. Supports OpenAI, Anthropic, Google, xAI (Grok), and DeepSeek with long context pricing, prompt caching, and reasoning tokens.

@traceloop/instrumentation-google-generativeai

Google Gen AI SDK Instrumentation

raindrop-ai

Primary Raindrop JavaScript SDK for event tracking, AI event ingestion, self-diagnostics, and tracing helpers.

@mai0313/vct

Vibe Coding Tracker - AI coding assistant telemetry/usage parser, aggregate JSONL events into CodeAnalysis results

@tanstack/solid-ai-devtools

Solid Devtools plugin for inspecting TanStack AI chat messages, tool calls, streams, and errors.

@fullstory/browser

The official Fullstory browser SDK

@statsig/statsig-ai

> ⚠️ **Beta Notice** > > This SDK is currently in **beta**. All APIs are **experimental** and subject to change. > Please review the [release notes](https://github.com/statsig-io/statsig-ai-node/releases) for any **breaking changes** before upgrading.

@vcueto/agent-viz

Real-time visualizer for Claude Code and GitHub Copilot CLI sessions: live event stream, token tracking, multi-agent topology with per-source badges, plus a token observatory built on the bundled netgain analysis engine.

@multi-agent/logger

@multi-agent/logger

pi-agentarium

Ambient multi-agent observability for Pi — a terminal habitat for coding agents.

@swarmtools/evals

Evaluation suite for swarm-tools multi-agent coordination

@nicofains1/agentwatch

Multi-agent observability: cascade failure detection, heartbeats, and forensic replay

claude-terminal-prism

PRISM — Agentic visibility layer for Claude. Intercepts, records, and replays multi-agent sessions.

@rifft-dev/rifft

Cross-framework debugger SDK for multi-agent AI systems

agenthud

CLI tool to monitor agent status in real-time. Works with Claude Code, multi-agent workflows, and any AI agent system.

@msbel/openclaw-aegis-signer

Ed25519-signed, SHA-256-chained tool-call audit log for OpenClaw multi-agent setups. Tamper-evident provenance chain.

agentfootprint

The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.

@tonerow/agent-framework

A framework for capturing, benchmarking, and visualizing agent goals, implementations, and runs.

@maiar-ai/monitor-console

Maiar: A Composable, Plugin-Based AI Agent Framework

@inkeep/agents-eval-api

Agents Evaluation API for Inkeep Agent Framework - handles evaluations, datasets, and evaluation runs

benchclaw-integrations

Adapters that connect any AI agent framework (JS/TS) to the P2PCLAW BenchClaw benchmark leaderboard.

@openclaw-agent-trace/sdk

Lightweight SDK for agent observability. Install in your agent framework to capture events.

agent-tail-core

Core utilities for agent-tail log capture plugins.

@axlsdk/eval

Evaluation framework for Axl agentic workflows

@agentica/benchmark

Agentic AI Library specialized in LLM Function Calling

@sebastianmattar/loopscore

Benchmark agentic coding AIs against coding tasks

@railtownai/railtracks-visualizer

A visualizer for Railtracks agentic flows

neatlogs

AI agent debugging, collaboration, and trace observability. Built for teams using CrewAI, OpenAI, and more.

trickle-cli

Zero-code runtime observability for JS/Python + AI agent debugging. Traces LangChain, CrewAI, OpenAI, Anthropic, Gemini. Eval, security, compliance, cost tracking. Free, local-first.

evalview

Open-source testing and regression detection framework for AI agents. Golden baseline diffing, CI/CD integration, works with LangGraph, CrewAI, OpenAI, Claude, HuggingFace, Ollama, and MCP.

@spy-llm/sdk

Framework-agnostic agent observability & LLM tracing. Supports OpenAI, CrewAI, AutoGen, LangGraph, and OpenTelemetry.

@atrib/openinference

OpenTelemetry SpanProcessor for atrib's verifiable action layer. Emits signed records from OpenInference-shaped spans.

agentflow-core

Monitor any AI agent system. Auto-detects failures, sends alerts, audits OS processes. Zero config, zero dependencies.

More in Agent frameworks & infrastructure