langfuse / langfuse
๐ชข Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
langfuse / langfuse
promptfoo / promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
gitkraken / vscode-gitlens
Supercharge Git inside VS Code and unlock untapped knowledge within each repository โ Visualize code authorship at a glance via Git blame annotations and CodeLens, seamlessly navigate and explore Git repositories, gain valuable insights via rich visualizations and powerful comparison commands, and so much more
silverbulletmd / silverbullet
The malleable knowledge base for you and your team. Powered by Markdown and Lua.
Helicone / helicone
๐ง Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 ๐
LuanRT / YouTube.js
A JavaScript client for YouTube's internal API, known as InnerTube.
langwatch / langwatch
Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
lmnr-ai / lmnr
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
ianarawjo / ChainForge
An open-source visual programming environment for battle-testing prompts to LLMs.
openlit / openlit
OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.
handsontable / hyperformula
HyperFormula is an open-source headless spreadsheet for business web apps. It comes with over 400 formulas, CRUD operations, undo-redo, clipboard support, and sorting.
Hexagon / croner
Trigger functions or evaluate cron expressions in JavaScript or TypeScript. No dependencies. Most features. Node. Deno. Bun. Browser.
MCPJam / inspector
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
deer-flow / llm-space
A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.
huggingface / aisheets
Build, enrich, and transform datasets using AI models with no code
google / adk-js
An open-source, code-first Typescript toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
roblox-ts / roblox-ts
A TypeScript-to-Luau Compiler for Roblox
wangmiaozero / pi-harness
The operational superset of Pi Coding Agent โ everything Pi, plus observability, governance, recovery, evaluation and multi-agent orchestration. Pi Coding Agent ็่ฟ็ปด่ถ ้โโๅฎๆด็ปงๆฟ Pi ็ๆ ธๅฟ่ฝๅ๏ผๅนถ่ฟไธๆญฅๆฉๅฑๅฏ่งๆตๆงใๆฒป็ใๆ ้ๆขๅคใ่ฏไผฐๅๅค Agent ็ผๆใ
darkrishabh / agent-skills-eval
A test runner for agentskills.io-style AI agent skills
LuanRoger / electron-shadcn
:electron: Electron Forge with shadcn-ui (Vite + Typescript)
hyoo-ru / mam_mol
$mol - fastest reactive micro-modular compact flexible lazy ui web framework.
comet-ml / opik-openclaw
๐ฆ Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more.
emo-xiaoyu / harness-mix
โ
jiggzson / nerdamer
a symbolic math expression evaluator for javascript
aws-samples / well-architected-iac-analyzer
Sample Generative AI tool for evaluating Infrastructure as Code and architecture diagrams against AWS Well-Architected best practices.
with-pebbly / aseprite-ai-artist
Your coding agent paints pixel art in the Aseprite window you already have open. MCP server + skills for Claude Code, omp, Codex, Gemini CLI and Cursor โ draws, looks at its work, animates and exports. Gallery and benchmark: pixeli.pebbly.space
ukanwat / overtime
Give a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.
dodoguardai / dodoguard
DodoGuard: Take AI Agents to Production. Test, validate, and operate AI Agents before and after deployment.for Dify, Coze, FastGPT, n8n, Tencent ADP, ByteDance HiAgent, and more.
Continuum-AI-Corp / OrcaReplay
OrcaReplay โ Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.
alanhuangyoo / crux
Taking the pi coding agent to Claude Code-level performance: 0.539 โ 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model, by re-engineering the agent.
suncityldp / zx-bench
A reproducible local LLM benchmarking platform with versioned datasets, deterministic scoring, and auditable reports.
S4US / Roqer
An open-source AI agent for Roblox Studio. Builds, scripts, and playtests in your place using your ChatGPT/Claude subscription or custom API.
hancao97 / hanai-investment-dsh
Local-first A-share research workbench for DeepSeek Harness: market dashboards, watchlists, valuation, four investor agents, versioned reports, and continuous post-report chat.
mrtinkz / ai-charter
Universal AI Charter: plain-language governance principles and a public certification registry for AI models, built on disclosure instead of a development slowdown.
AgentX-ai / AgentX-Trace-Eval
The trust layer for your AI agent on any platform. Trace and evaluate your agent. Full Observability. Run Model portability analysis. Autotune your agent.
deeplethe / fde-gym
The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
azbigboss66-oss / skillfoo-v3
Bounded, evidence-driven SKILL.md evolution under frozen evaluation and safety contracts.
shift-labs-ai / pi-rlm
A pi extension with a single tool: execute, running TypeScript in a persistent Bun evaluator. Files, shell, and subagents are expressed as code rather than as more tools.
Hona / openeval
Typed prompt-and-judge SDK, CLI, and evidence viewer for agent evaluations
composo-ai / DishwasherBench
A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.
review-bench / ReviewBench
ReviewBench is an open, reproducible benchmark for evaluating AI code review systems on real-world pull requests.
evaldock / EvalDock
An end-to-end evaluation framework for heterogeneous agents and plugins.
ryuxdsgn / design-intelligence
RYUX gives AI agents the design intelligence to understand product context, make intentional design decisions, and evaluate what they create.
DevMortimer / pi-typesafe
TypeSafe decisions for Pi: batched evaluation tool, terminal playground, and typed API for extension authors
oxlaboratory / oxis
๐ป OXIS is a Lua-scriptable programmable workspace for Windows & Linux โ combining a real terminal, code editor, automation, documents and plugins into one fully customizable app. No account. No telemetry.
luantak / is-malicious
A codebase scanner that helps you not run malicous code
aravelo7 / AgentPatchCheck
Coding Agent Runtime & Evaluation Harness for controlled repository-level software repair
microsoft / scope
Scope | Agentic Experience evaluation platform (Research preview)
lukstei / slop-grader
Jev-powered, rule-based grader for text files. Runs every rule against every line in parallel. No skimming, no missed lines.
jakemanger / whatmodel
Find the best AI models you can run
AlchemyInCode / plumloom-autoeval-oss
Open-source CLI and MCP server for evaluating AI applications and gating releases with reliability evidence.
smkrv / jev-calibrate
Calibrate Jev questions against your own labels: tune criteria on labelled examples, confirm on a held-out set, get a verdict per question. Unofficial.
SagarBiswas-MultiHAT / promptVault
The private, offline-first prompt manager for AI Agent power users. Features AES-256 encrypted local storage, a built-in AI Librarian to evaluate & refine prompts with Gemini & Groq fallback, dynamic {{variable}} templates, Supabase cloud sync, analytics, and instant โK search.