langfuse / langfuse
πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
langfuse / langfuse
comet-ml / opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
VoltAgent / voltagent
AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework
maximhq / bifrost
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200Β΅s of overhead at 5k RPS.
mnfst / llm-gateway
Open Source LLM Gateway π¦
Helicone / helicone
π§ Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 π
FailproofAI / failproofai
Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.
coze-dev / coze-loop
Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.
latitude-dev / latitude-llm
Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces.
pydantic / logfire
AI observability platform for production LLM and agent systems.
lmnr-ai / lmnr
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
openlit / openlit
OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.
future-agi / future-agi
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing Β· Evals Β· Simulations Β· Datasets Β· Gateway Β· Guardrails. Self-hostable. Apache 2.0.
jwuthri-cloud / Tracely-ai
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
JudgmentLabs / judgeval
The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.
ThinkWatchProject / ThinkWatch
Self-hosted AI API and MCP gateway for organizations: SSO and RBAC, per-user identity for MCP tool calls, PII redaction and tool-call guards, rate limits, budgets, cost accounting and audit logs.
traceroot-ai / traceroot
TraceRoot - Open Source AI Agent Improvement Platform YC S25
ThinkWatchProject / ThinkWatch-Lite
Local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux: switch upstreams without touching clients, keep API keys from relays, cut off malicious tool calls, trace every request.
comet-ml / opik-openclaw
π¦ Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more.
databufflabs / databuff
DataBuff is an AI-native APM built on OpentelemetryοΌwith multi-agent troubleshooting out of the box.
Abilityai / trinity
Self-hosted AI Agents Platform for AI Native companies. Claude Code, Codex, Gemini, OpenCode agents. Apache 2.0.
Justin0504 / Aegis
Runtime policy enforcement for AI agents. Cryptographic audit trail, human-in-the-loop approvals, kill switch. Zero code changes.
Continuum-AI-Corp / OrcaReplay
OrcaReplay β Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.
scasella / claude-flightdeck
A Claude Code mod that puts a live agent dashboard in your terminal: context and cost, advisor timeline, every permission check, subagent cards and swimlanes.
ewsun22 / codex-manager
Local-first desktop control center for OpenAI Codex β sessions, models, tokens, accounts, providers, and AGENTS.md.
duqaXxX / seedeep
What Claude Code is doing between your prompt and its answer, drawn live from the logs it already writes: every model call, every tool, each subagent on its own context window and its own model, and what the session produced. Local and read-only β your session content never leaves the machine.
talhaanwarch / awesome-llmops-platforms
Curated list of open-source LLMOps tools: tracing, evals, AI gateways, guardrails and prompt management. Sorted by stars, refreshed weekly.
loongsuite / dsh-plugin
OpenTelemetry tracing for DeepSeek Harness (dsh): turns each agent turn into a GenAI span tree β steps, LLM calls with TTFT, tool executions, token usage β exported over standard OTLP to Jaeger, Grafana Tempo, SigNoz, Langfuse, or any compatible backend.