promptfoo / promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
langfuse / langfuse
🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
MemTensor / MemOS
Self-evolving memory OS for LLM & AI Agents: ultra-persistent memory, hybrid-retrieval, and cross-task skill reuse, with 35.24% token savings and DeepSeek Harness support.
mastra-ai / mastra
Mastra is the modern TypeScript framework for AI-powered applications and agents.
labring / FastGPT
FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.
Phant0Meow / dsh-meow-memory
Cross-session memory plugin for DeepSeek Harness: seven-layer SQLite store (soul/user/project/fact/lesson/topic/rules), BM25 retrieval, per-window dream consolidation. 跨会话七层长期记忆插件。
microsoft / scope
Scope | Agentic Experience evaluation platform (Research preview)
openlit / openlit
OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.
dzhng / jevgrep
Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
neatlogs / neatroots
neatroots: see, check and replay your AI agent's runs on your machine while you build it. Local-first, by Neatlogs.
Marker-Inc-Korea / AutoRAG
Your Personal Librarian Agent. Ask anything on your computer, organize files, remember important info and files.
langwatch / langwatch
Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
lmnr-ai / lmnr
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
MCPJam / inspector
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
evaldock / EvalDock
An end-to-end evaluation framework for heterogeneous agents and plugins.
Helicone / helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
handsontable / hyperformula
HyperFormula is an open-source headless spreadsheet for business web apps. It comes with over 400 formulas, CRUD operations, undo-redo, clipboard support, and sorting.
Aas-ee / open-webSearch
Multi-engine MCP server, CLI, and local daemon for agent web search and content retrieval — skill-guided workflows, no API keys.
huggingface / aisheets
Build, enrich, and transform datasets using AI models with no code
google / adk-js
An open-source, code-first Typescript toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
darkrishabh / agent-skills-eval
A test runner for agentskills.io-style AI agent skills
prevalentWare / opencode-goal-plugin
OpenCode goal plugin for Codex-style goal mode, /goal slash commands, persistent objectives, and AI coding agent focus.
alexngdev99 / rork-medieval-3d-chess
Created by Rork
deeplethe / fde-gym
The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
donvito / local-evals
An eval app which runs locally and uses local models or any OpenAI-compatible API
review-bench / ReviewBench
ReviewBench is an open, reproducible benchmark for evaluating AI code review systems on real-world pull requests.
TanStack / query
🤖 Powerful asynchronous state management, server-state utilities and data fetching for the web. TS/JS, React Query, Solid Query, Svelte Query and Vue Query.
vercel / swr
React Hooks for Data Fetching
FailproofAI / failproofai
Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.
CaviraOSS / LongMemory
Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc.
ianarawjo / ChainForge
An open-source visual programming environment for battle-testing prompts to LLMs.
Hexagon / croner
Trigger functions or evaluate cron expressions in JavaScript or TypeScript. No dependencies. Most features. Node. Deno. Bun. Browser.
Kong / swrv
Stale-while-revalidate data fetching for Vue
samchon / nestia
NestJS Helper + AI Chatbot Development
deer-flow / llm-space
A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.
volcengine / SearchCLI
Open CLI for integrating AI search, recommendation, and conversational retrieval into agent systems and business systems
jdevalk / specification.website
Website specification — HTML, accessibility, security, SEO, agent-readiness. Platform-agnostic, sourced, MIT.
hyoo-ru / mam_mol
$mol - fastest reactive micro-modular compact flexible lazy ui web framework.
mgechev / skillgrade
"Unit tests" for your agent skills
comet-ml / opik-openclaw
🦞 Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more.
xynehq / xyne
AI-first Search & Answer Engine for work. Open-source alternative to Glean.
felladrin / MiniSearch
Minimalist web-searching platform with an AI assistant that runs directly from your browser. Demo: https://felladrin-minisearch.hf.space
soruly / trace.moe-telegram-bot
This Telegram Bot can tell the anime when you send an screenshot to it
jiggzson / nerdamer
a symbolic math expression evaluator for javascript
awslabs / generative-ai-cdk-constructs
AWS Generative AI CDK Constructs are sample implementations of AWS CDK for common generative AI patterns.
aws-samples / well-architected-iac-analyzer
Sample Generative AI tool for evaluating Infrastructure as Code and architecture diagrams against AWS Well-Architected best practices.
ukanwat / overtime
Give a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.
Continuum-AI-Corp / OrcaReplay
OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.
dodoguardai / dodoguard
DodoGuard: Take AI Agents to Production. Test, validate, and operate AI Agents before and after deployment.for Dify, Coze, FastGPT, n8n, Tencent ADP, ByteDance HiAgent, and more.
a86582751 / dsh-nexttavern
DeepSeek Harness 长篇角色扮演agent（DSH酒馆插件）：SillyTavern 角色卡导入、行动选项卡、分支对话管理、长篇记忆、关键词与语义混合检索、交互式角色卡创作、世界书、多角色 Agent 集群、文风预设、小说与角色卡导出、一键安装。 / Roleplay Agent 、Tavern plugin for DSH (DeepSeek Harness): branch conversation, long-term memory, hybrid keyword and semantic retrieval, SillyTavern cards, interactive card authoring, card export, one-click installers.
alanhuangyoo / crux
Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with the same self-hosted 27B model, by re-engineering the agent.
1173591564 / Dynamics-memory
LLM agent 的有界自纠错长期记忆层：把连续、含矛盾的项目交互日志蒸馏成有界、可自更新的记忆库。Bounded, self-correcting long-term memory for LLM agents — candidate pool + value dynamics + tension adjudication + causal-replay evals.
suncityldp / zx-bench
A reproducible local LLM benchmarking platform with versioned datasets, deterministic scoring, and auditable reports.
wangmiaozero / pi-harness
The operational superset of Pi Coding Agent — everything Pi, plus observability, governance, recovery, evaluation and multi-agent orchestration. Pi Coding Agent 的运维超集——完整继承 Pi 的核心能力，并进一步扩展可观测性、治理、故障恢复、评估和多 Agent 编排。
vercel-labs / run
A secure and efficient alternative to `eval` for coding agents
mrtinkz / ai-charter
Universal AI Charter: plain-language governance principles and a public certification registry for AI models, built on disclosure instead of a development slowdown.
AgentX-ai / AgentX-Trace-Eval
The trust layer for your AI agent on any platform. Trace and evaluate your agent. Full Observability. Run Model portability analysis. Autotune your agent.
azbigboss66-oss / skillfoo-v3
Bounded, evidence-driven SKILL.md evolution under frozen evaluation and safety contracts.
tntexploding / StruInfo
Self-hosted, traceable document-to-knowledge workspace with deterministic workflows, optional AI, search, RAG, and an editable knowledge graph.
shift-labs-ai / pi-rlm
A pi extension with a single tool: execute, running TypeScript in a persistent Bun evaluator. Files, shell, and subagents are expressed as code rather than as more tools.