123Cx330Yrx / riftcoach-agent

A quality-gated League of Legends post-game coaching agent built on deterministic match data, RAG, and LLM evaluation.

openlayer-ai / jevals

Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.

InkMind-AI / Dyn-3D

TempoVista is the official implementation of Dyn-3D, a benchmark and training pipeline for 3D spatial reasoning in vision-language models. The release contains frame-selection tools, SFT and Kinematic-GSPO training code, benchmark evaluation scripts, Dyn-3D annotations, and checkpoints for Qwen3-VL and InternVL3.5.

dlgjr / FINAR-VL

Open-source financial multimodal LLM for document understanding, chart/table reasoning, numerical reasoning, and financial analysis, with SFT, dual-branch RL, MOPD distillation, and public benchmark evaluation.

Simreal-AI / Xitadel-QuantBench

[Public preview] The first public quant trading benchmark scored against what professional traders actually made on the same asset

Simreal-AI / MathmoBench

[Public preview] An advanced AI Benchmark for math, produced by mathmo at St John's College, Cambridge.

ABCLDZ / open-world-research-reasoning-eval

Public reproducibility bundle for open-world deep-research reasoning evaluation

Simreal-AI / hard-puzzle-benchmark

[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.

thiientv / godmode

Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, and evals.

SalesforceAIResearch / Beagle

:dog: Beagle is designed to be the unified platform for the broader Recursive Self-Improvement (RSI) ecosystem. Think of it as Hugging Face Trainer for agent evolution. Its efficient rollout engine enables evaluation and evolution at scale across different agents, datasets, benchmarks, and evolution methods.

MaxwellCCC / hangzhou-mahjong

Hangzhou Mahjong training and evaluation framework with public-information Transformers, paired branch reinforcement learning, and the r2600 checkpoint.

NocodeMrLi / mini-program-engineering-skill-suite

Mini Program Engineering Skill Suite is an Agent Skill suite for evidence‑first mini‑program development. It helps agents bring WeChat or other mini‑program projects from vague intent to reliable engineering work: project intake, product specification, architecture, implementation, UI & device adaptation, debugging, verification, release readiness.

tttboy123 / jlens-evolve

JLens Evolve v3: evidence-centric evolution platform (autonomous Teacher + local Qwen + official SWE-bench native evaluation). Branch codex/v3-evolution-10000 carries transport resilience, Router-key normalization, per-round temperature beam, and baseline fail-soft.

harish-gaggar / mcp-server-gateway

MCP gateway & OAuth MCP servers. LangGraph agent: memory, eval/tracking, PII guard, context trimming. Docker quick start for enterprise agents.

sangelege / forestfix

Verifiable multi-agent patch evaluation infrastructure

OpenMOSS / SWE-bench-Science

A benchmark for testing whether coding agents can resolve engineering tasks in scientific software

WaseemGhanem98 / AgentCheck

Behavioral testing for AI agents.

ZJU-REAL / ageval

Agent eval on one running base. Swap the agent under test with plugins; run the same dataset anywhere.

RenaGao / jev-dataops

An open-source JEV-powered workbench for streaming data selection, quality evaluation, automatic LoRA training and held-out model evaluation.

mallahyari / nemotron-asr-streaming-farsi

Streaming Persian (Farsi) speech recognition: fine-tuning NVIDIA Nemotron 3.5 ASR streaming. Data prep, training, evaluation, inference.

AI-Safety-Institute / transect

Transcript analysis for long-horizon agent evaluations

Kiln-AI / Seahaven

A framework for building synthetic worlds for AI agents. Useful for RL and evaluation environments.

david-g-3654 / homebench

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

wanqili857-byte / fictionforge

多智能体小说写作框架 | Multi-agent novel-writing framework — 每个角色都有自己的脑子,每一章都有质量门禁

BROccoLi-921 / AI-testing-evaluation

—

WalrusQuant / sports-analytic-skills

Multi-sport data science Agent Skills library. Portable judgment for sports modeling — doctrine, validation, leakage control, and honest evaluation.

bodepudimuneendra-netizen / laya-jev-GraphRAG

A database-agnostic Agentic GraphRAG framework using swappable System One models (local Laya / cloud Jev). A plug-and-play intelligence layer featuring a complete 4-phase pipeline, continuous evaluation and custom A* traversal for any graph database.

pipelinelabai / paperbench-expanded

PaperBench High-Difficulty and Expanded Tasks: paper reproduction beyond machine learning. 93 tasks in 12 research areas, run in domain-native software and graded by clean replay. 12 public tasks.

maoyadongsh / siq-agent-security

Research artifacts for provenance-bound agent authorization, signed effect evidence, and reproducible security evaluation.

YuanKJing / Jev-as-Policy

The highly anticipated open-source repository for JEV as Policy enables one-click setup of the simulation environment. Evaluations of Astra + JEV on benchmarks such as RoboTwin will also be released soon.

RookieZhou0617 / risk-aware-financial-fraud-detection

Multi-source risk-aware graph learning for financial statement fraud detection under strict temporal evaluation.

PyModel / jev-judge-mcp

Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.

AhmadJeddi / Plant-Disease-Classification

A ResNet50-DCBAM deep learning framework for plant disease classification using the PlantVillage dataset with reproducible training and evaluation pipelines.

xixiaouab / ReaLVR-code

ReaLVR: Grounding latent visual reasoning in visual evidence. Training and evaluation code.

campusx-official / rag-eval-deepeval

Rag evaluation using DeepEval

datamllab / A2E

—

TapnTell-AI / openniw

Open-source AI-assisted NIW/EB1A/O1 petition preparation — from free evaluation to filing package. Not legal advice.

VectorSpaceLab / AgentSWE

Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimization.

Qin-sor / agent-context-lab

A minimal coding-agent harness built to measure what context management actually costs: constraint survival after compaction (8%→83%), default-FAIL verification against false completions, and KV-cache prefix design (21%→90%). Stdlib-only, real deepseek-chat runs

noyce6983981-max / ocr-vlm-local-retrieval

Local-first OCR-VLM multimodal document retrieval and independent evaluation prototype.

zjunlp / AutoSciRub

Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

nickname21kmr / gameops-investigator

Evidence-first game operations incident investigation agent with read-only MCP tools and reproducible evaluations.

inworld-ai / open-tts-eval

Open, reproducible TTS/STT evaluation toolkit: WER/CER via Whisper ASR, NISQAv2, audio health, and prosody metrics with offline black-and-white reports.

hastimal / gemini-healthcare-agentic-platform

Beyond RAG: An open-source agentic healthcare search framework using Gemini and Google ADK for query fan-out, evidence retrieval, grounding, citation evaluation, and trustworthy healthcare information discovery.

theaiautomators / jev-arena

A local evaluation lab for decision models. Compare accuracy, speed and memory requirements through an interactive dashboard, inspectable test cases, workflow replays and shareable reports.

m2dumpling / pawbot

A self-hosted AI agent runtime with WebUI, MCP tools, context governance, and Record & Replay for deterministic execution & evaluation.

talhaanwarch / awesome-llmops-platforms

Curated list of open-source LLMOps tools: tracing, evals, AI gateways, guardrails and prompt management. Sorted by stars, refreshed weekly.

tawdesangeeta1973-coder / ptcg-belief-mcts

Information-Set MCTS-inspired agent and reference simulation for the Pokémon TCG AI Battle Challenge — belief-based decision policy, deck architecture, and evaluation harness for the Strategy Category submission.

ydflow / research-trail

研迹 ResearchTrail｜本地 AI Native投资研究工作台。串联证券、组合、证据报告、论点复审与 Today 简报，让研究有据、判断留痕。Python 业务内核 · Electron 桌面 · 可复现 Agent 评测。

Gen-Verse / PAST-Bench

PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

hawaiicoco / slotvox

End-to-end spoken language understanding toolkit: a small trainable neural audio codec with RVQ, joint intent-and-slot models over speech features, streaming partial intents, a speech-LLM instruction data pipeline, and turn-level evaluation with reproducible reports. NumPy core with a CPU-only torch extra; synthetic data only, fully offline.

WenjieShu / SolveEdit

SolveEdit-Plan and evaluation code for SolveEdit

gwendolynashe / llm-evaluation-workbench

Argolink-focused LLM evaluation and regression workflow for GPT-6 Sol.

ArcInstitute / cell-eval2

GPU-accelerated evaluation of perturbation prediction models

zsxm1998 / awesome-perception-aware-rlvr

Unified reproductions, benchmarking & evaluation for perception-aware RLVR and OPD in VLMs. 19 methods · 30+ benchmarks · Official code of CGPO (ACM MM 2026 Oral).

Grit-77 / done-is-a-claim

Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.

ZhuWenjie98 / MMOOC

MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models

Human-Agent-Society / reef-eval

🌊 Evaluation infrastructure for self-evolving agents, on the Harbor task standard.

LGAI-Research / PCBWorld

A reinforcement learning environment for PCB autorouting, built on KiCad's push-and-shove router. Includes a decoder-only PPO/GRPO agent, LLM tool-calling and rule-based baselines, and a uniform 3-stage evaluation pipeline. The routing engine ships separately as the PCBWorld-Engine submodule (GPLv3).

PillCrew / deedchain

Benchmark of report fidelity for browser agents: does the agent's final report tell the truth about what it actually did?