ifixai-ai / iFixAi
Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
Giskard-AI / giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents
coze-dev / coze-loop
Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.
truera / trulens
Evaluation and Tracking for LLM Experiments and AI Agents
future-agi / future-agi
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
mozilla-ai / any-agent
A single interface to use and evaluate different agent frameworks
jwuthri-cloud / Tracely-ai
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
TIGER-AI-Lab / ClawBench
Open-source benchmark for browser AI agents on daily tasks.
benchflow-ai / awesome-evals
A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
chirpz-ai / pandaprobe
open source agent engineering platform: traces, evals, and metrics to debug and improve your AI agents. Integrates with LangGraph, CrewAI, Claude Agent SDK, and more.
keli-wen / agy-staff
Hire Google's Antigravity CLI (agy) as a fast Gemini staffer for Claude Code and OpenAI Codex.
NVIDIA / SkillEvaluator
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
scaleapi / agentenv-framework
Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.
zgcagi / Harness-RSI
Harness self-evolution: update targets, theory, evaluation, and reported evidence. Bilingual reading list and project website.
i3T4AN / KADATH
Evolutionary multi-agent runtime that breeds, evaluates, and improves autonomous agents across reproducible epochs to converge on optimization of a goal.
roy-tong / AgentMeasure
Inspect Codex / Claude logs, verify metering samples and re-count supported billing exports. Free local tools; bounded $10 diagnosis and $20 QA.
alanhuangyoo / crux
Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model, by re-engineering the agent.
deeplethe / fde-gym
The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
thiientv / godmode
Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, and evals.
WaseemGhanem98 / AgentCheck
Behavioral testing for AI agents.
datamllab / A2E
—
VectorSpaceLab / AgentSWE
Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimization.
Grit-77 / done-is-a-claim
Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.