Kautenja / gym-super-mario-bros
Gymnasium environments for NES Mario games, with task metadata, reward instrumentation, and reproducible evaluation interfaces.
Kautenja / gym-super-mario-bros
nolabs-ai / deepfabric
Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline
flosell / trailscraper
A command-line tool to get valuable information out of AWS CloudTrail
groq / openbench
Provider-agnostic, open-source evaluation infrastructure for language models
chirpz-ai / pandaprobe
open source agent engineering platform: traces, evals, and metrics to debug and improve your AI agents. Integrates with LangGraph, CrewAI, Claude Agent SDK, and more.
claw-eval / claw-eval
Claw-Eval is an evaluation harness for evaluating LLM as agents. All tasks verified by humans.
huggingface / upskill
Generate and evaluate agent skills for code agents like Claude Code, Open Code, OpenAI Codex
MigoXLab / dingo
Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
harbor-framework / terminal-bench-science
Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains
hailo-ai / hailo_model_zoo
The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment
xmarre / ComfyUI-Spectrum-MiniMax-H3
Training-free Spectrum acceleration for ComfyUI’s native MiniMax H3 audio-video model. Uses Chebyshev ridge feature forecasting to skip selected H3 transformer evaluations, with adaptive scheduling, sampler-aware support for Euler, ER-SDE, RES, SEEDS and SA-Solver, CPU/VRAM history storage, and fail-closed native fallbacks.
EverMind-AI / SkillCorpus
Open-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.
probabl-ai / skore
Track your Data Science. Skore's open-source Python library accelerates ML model development with automated evaluation reports, smart methodological guidance, and comprehensive cross-validation analysis.
MIC-DKFZ / nnDetection
nnDetection is a self-configuring framework for 3D (volumetric) medical object detection which can be applied to new data sets without manual intervention. It includes guides for 12 data sets that were used to develop and evaluate the performance of the proposed method.
ServiceNow / AgentLab
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
ggozad / haiku.rag
Agentic RAG for local and self-hosted document search: hybrid retrieval, reranking and multimodal RAG on embedded LanceDB, with Docling parsing and an MCP server
HowieHwong / TrustLLM
[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
KRLabsOrg / LettuceDetect
Span-level grounding verification for RAG, code, and tool-grounded AI outputs.
locuslab / open-unlearning
[NeurIPS D&B '25] The one-stop repository for LLM unlearning
danthedeckie / simpleeval
Simple Safe Sandboxed Extensible Expression Evaluator for Python
run-llama / ParseBench
ParseBench - A Document Parsing Benchmark for AI Agents
google / seqio
Task-based datasets, preprocessing, and evaluation for sequence models.
feicong / re-docs
reverse engineering docs.
luar123 / zigbee_esphome
External zigbee component
dbt-labs / dbt-project-evaluator
This package contains native dbt checks to report and fix issues in your dbt project
aws-samples / service-screener-v2
A tool for customers to evaluate their AWS service configurations based on AWS and community best practices and receive recommendations on potential improvements.
facebookresearch / meta-agents-research-environments
Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
Mic92 / nix-fast-build
Combine the power of nix-eval-jobs with nix-output-monitor to speed-up your evaluation and building process.
skygazer42 / MimirQ
中文优先的企业 RAG 知识库：可控解析、治理、切块、混合检索、重排、引用、图谱、评测与 Dify 接入。
OmniJev / awesome-jev-gallery
🔥🔥 Awesome Jev: open-source models, projects, benchmarks, and independent evaluations for Jev and System One models.
uoftcprg / pokerkit
An open-source Python library for poker game simulations, hand evaluations, and statistical analysis
official-stockfish / nnue-pytorch
Stockfish NNUE (Chess evaluation) trainer in Pytorch
OpenMOSS / EasyWAM
A unified framework for training, fine-tuning, and evaluating World Action Models
ChidiebereNjoku / ModelBrief
An open-source Python library that automates the analysis and reporting of machine learning experiments, generating structured and comprehensive reports that make model performance, results, and key insights easier to understand and evaluate by data professionals, with optional Groq-powered AI explanations and recommendations for deeper insights.
open-daimax / daimax-appbench
An automated benchmark platform for AI-generated apps — built-in evaluation rules, standard sample sets, and layered test cases across a multi-dimensional metric system. One pipeline runs build, install, E2E tests, and scoring into a shareable report.
aaron-for-value / VeriRun
Evidence-first infrastructure for reproducible, isolated executable evaluation and online rewards.
fidetolabs / qanat
A data platform for investment research. Ingests financial data sources and computes point-in-time financial metrics for valuation and screening. Delivered as an MCP server.
MirroS-Lab / HarnessEval-W
HarnessEval-W: Agentifying the Evaluation of Visual Worlds
ktwu01 / benchmark-radar
Track 33,707+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.
jinzijian / EvoTrace
Compile real-world Claude Code and Codex trajectories into verified, tradable post-training assets.
xiaobright / modeltest
Personal LLM engineering-maintenance evaluation harness (V4.1b, frozen). Not a public benchmark.
i3T4AN / KADATH
Evolutionary multi-agent runtime that breeds, evaluates, and improves autonomous agents across reproducible epochs to converge on optimization of a goal.
waiterve / wai-play
WAI Play - AI web game testing and quality evaluation platform
TriWorldBench / TriWorldBench
A Benchmark Evaluating Triple-View Embodied World Models
roy-tong / AgentMeasure
Inspect Codex / Claude logs, verify metering samples and re-count supported billing exports. Free local tools; bounded $10 diagnosis and $20 QA.
MoonshotAI / PerceptionBench
PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
jsdhwfmax / EvalForge
Evaluator-neutral AI evaluation evidence, baseline regression gates, and JSON, JUnit, and SARIF reports for CI.
EastForce / worker-ai-spark
开源劳动者AI研究与评测项目：建设劳动知识、公开评测基准与共同治理机制｜Open worker-centered AI research and evaluation project.
KhanCold / merchantbench
MerchantBench is a 365-day, order-level benchmark for evaluating the long-term coherence of LLM agents in seller-side e-commerce operations.
solis-team / XRepoTest
[EMNLP 2026] A multilingual benchmark for repository-level unit test generation, evaluating LLMs across 5 programming languages and real-world code under reproducible, Docker-based execution.
nssmd / RoboRSI
Robot-agent harness with a CLI and local Web console for LIBERO short evaluation
AMAP-ML / LoopArena
Benchmarking models as runtime Controllers for Loop Engineering
Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
MrZoyo / deslop-GPT
Deletion-first Agent Skill for removing test bloat, verification theater, and speculative fallbacks while preserving behavior.
salikhussain71-code / PAKGOV-RAG-project
Research project on bilingual Urdu-English retrieval-augmented generation for Pakistani government and legal documents, with OCR, hybrid retrieval, citation grounding, and evaluation.
fengzhizi715 / YoloWebAgentStarter
Local-first YOLO workbench for annotation, training, evaluation, and AI assistance.
LINs-lab / UltraText_Bench
UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation
ernie-research / FoR-T2I
Official repository of FoR-T2I, a benchmark evaluating T2I models from a different perspective.
ConstantineChenn / SakikoMind
Enterprise AI customer-service Agent platform featuring hybrid intent recognition, multi-agent orchestration, MCP-powered RAG, dynamic Skills, memory, human handoff, observability and evaluation. 小祥智能客服系统，支持可追溯、可观测与云端部署。
To3akaRin / VLMForge
Evaluate visual models on your own images, JSON Schema, and production constraints with LangGraph, Pareto analysis, and a no-key Replay demo.