m2dumpling / pawbot
A self-hosted AI agent runtime with WebUI, MCP tools, context governance, and Record & Replay for deterministic execution & evaluation.
m2dumpling / pawbot
talhaanwarch / awesome-llmops-platforms
Curated list of open-source LLMOps tools: tracing, evals, AI gateways, guardrails and prompt management. Sorted by stars, refreshed weekly.
tawdesangeeta1973-coder / ptcg-belief-mcts
Information-Set MCTS-inspired agent and reference simulation for the Pokémon TCG AI Battle Challenge — belief-based decision policy, deck architecture, and evaluation harness for the Strategy Category submission.
ydflow / research-trail
研迹 ResearchTrail｜本地 AI Native投资研究工作台。串联证券、组合、证据报告、论点复审与 Today 简报，让研究有据、判断留痕。Python 业务内核 · Electron 桌面 · 可复现 Agent 评测。
Artem76228 / PaperOS
A whole little OS for the M5Stack Cardputer - launcher, file manager, paint, browser, offline AI, game emulator, Lua scripting, and more.
Gen-Verse / PAST-Bench
PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents
hawaiicoco / slotvox
End-to-end spoken language understanding toolkit: a small trainable neural audio codec with RVQ, joint intent-and-slot models over speech features, streaming partial intents, a speech-LLM instruction data pipeline, and turn-level evaluation with reproducible reports. NumPy core with a CPU-only torch extra; synthetic data only, fully offline.
WenjieShu / SolveEdit
SolveEdit-Plan and evaluation code for SolveEdit
gwendolynashe / llm-evaluation-workbench
Argolink-focused LLM evaluation and regression workflow for GPT-6 Sol.
ArcInstitute / cell-eval2
GPU-accelerated evaluation of perturbation prediction models
mark-clarkops01e0 / FiveM-Ultimate-Assistant
Ultimate assistant for FiveM. Features: Money Hack, Aimbot, ESP (Player/Item/Vehicle), Wallhack, Speed Hack, Fly/Noclip, Teleport, Lua Executor, Vehicle Spawner, God Mode, and HWID Spoofer. 25+ functions. Undetected.
HongyeYangGT / Part_HackBench
A stress-testing benchmark for partial-completion evaluators in stateful multi-turn agents, focusing on temporary progress, rollback, and attribution failures.
ZeyuChang0471 / marvel
Multi-Agent Research & Valuation Engine for Long-short A股多Agent投研分析系统 9位AI分析师 → 质量门控 → 多空辩论 → 风控评估 → 最终决策
zsxm1998 / awesome-perception-aware-rlvr
Unified reproductions, benchmarking & evaluation for perception-aware RLVR and OPD in VLMs. 19 methods · 30+ benchmarks · Official code of CGPO (ACM MM 2026 Oral).
Grit-77 / done-is-a-claim
Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.
ZhuWenjie98 / MMOOC
MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
Human-Agent-Society / reef-eval
🌊 Evaluation infrastructure for self-evolving agents, on the Harbor task standard.
HERRY423 / BioNexus
Scientific reliability layer for AI-assisted biology: evaluates what evidence warrants, caps unsupported claims, audits analytical validity, and prevents silent scientific overreach.
LGAI-Research / PCBWorld
A reinforcement learning environment for PCB autorouting, built on KiCad's push-and-shove router. Includes a decoder-only PPO/GRPO agent, LLM tool-calling and rule-based baselines, and a uniform 3-stage evaluation pipeline. The routing engine ships separately as the PCBWorld-Engine submodule (GPLv3).
datosh / nixos-vex
OpenVEX feed and NixOS module to manage NixOS CVEs, evaluated against your system's configuration.
luanmorenommaciel / brief-spec
Type-aware, evidence-backed delivery contract for AI coding harnesses. Different agents in. One predictable human handoff out.
PillCrew / deedchain
Benchmark of report fidelity for browser agents: does the agent's final report tell the truth about what it actually did?
gulucaptain / MiniMax-H3-Reason
[Evaluation] Can Minimax-H3 Reason About the Physical World?
YellowPancake / quant-autoresearch
A minimal framework for Agents to research quantitative strategies under a fixed evaluation protocol.
Zaious / jev-capability-atlas
Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙語 repo。
sumleo / RLCDAlignBench
RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)
Archive228 / adversarial-contract-gate
Self-evaluation is a trap — grade against a contract with a separate evaluator. Built from Anthropic's 'Build Agents That Run for Hours' (39:00). No deps, ./run.sh, 9 tests.
parthmax2 / ghostrun
pytest for AI Apps — Fast, free, offline testing and self-improving prompts for Python developers.
NEDONION / evalhub
An LLM Benchmark evaluation platform for automated model testing and analysis. 面向大语言模型的 Benchmark 自动化评测平台，支持模型测试与结果分析
volt-l18 / llm-redteam-pipeline
An automated, multi-turn LLM red-teaming pipeline that pits an Attacker model against a Target model to evaluate safety boundaries, featuring LLM-as-a-Judge scoring and an interactive HTML vulnerability dashboard.
LUMIA-Group / ncp_olmo_eval
Official vLLM evaluation for OLMo and NCP-ArchPreview: GSM8K, SciQ, Core88, RULER, and HELMET
Odyphus / research-decision-skill
基于 Price、Wieman 等人 29 Decisions 研究的科研决策 Skill：结合已有结果、可追溯方法与领域资料，解释下一步为什么值得做。支持多种 Agent Skills 宿主。Research decision support for STEM.
francordel / myrtia
Myrtia — agent-native red teaming for LLMs and AI agents: a deliberating council of attacker models (propose, cross-critique, Borda vote, chairman synthesis), 13 attack strategies, live turn-by-turn UI and ASR analytics. Open-source, built on IBM ARES.
tianyi-zhang-02 / cooking-agi
Learning how to cook AGI from models, data, memory, search, feedback, and evaluation.
cyptnpassira / PromptLedger
PromptLedger - version prompt templates, run deterministic comparison snapshots, score candidate responses, and ship diffable experiment reports. Python core plus a Go companion CLI.
tomiholtz / s3-files-ai-sdk
Give Vercel AI SDK agents a real persistent filesystem with Amazon S3 Files.
DeepCybo-PhysAI / PhysBrainEvalKit
A reproducible evaluation toolkit on spatial and embodied-intelligence benchmarks.
AbdelStark / jev-benchmarks
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
allenai / reward-bench
RewardBench: the first evaluation tool for reward models.
Simreal-AI / Simreal-MLBench
[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.