vikashplus / robohive
A unified framework for robot learning
vikashplus / robohive
HowieHwong / TrustLLM
[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
IPNL-POLYU / UrbanNavDataset
UrbanNav:An Open-sourced Multisensory Dataset for Benchmarking Positioning Algorithms Designed for Urban Areas
locuslab / open-unlearning
[NeurIPS D&B '25] The one-stop repository for LLM unlearning
run-llama / ParseBench
ParseBench - A Document Parsing Benchmark for AI Agents
SalesforceAIResearch / MCP-Universe
MCP-Universe is a comprehensive framework designed for RL training, benchmarking, and developing AI agents for general tool-use.
Tencent-Hunyuan / CL-bench
CL-bench: A Benchmark for Context Learning
facebookresearch / meta-agents-research-environments
Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
zli12321 / LHTB
Long Horizon Terminal Benchmark with Dense Reward Grading
itbench-hub / ITBench
An open source benchmarking framework for IT automation
OmniJev / awesome-jev-gallery
🔥🔥 Awesome Jev: open-source models, projects, benchmarks, and independent evaluations for Jev and System One models.
liu00222 / Open-Prompt-Injection
This repository provides a benchmark for prompt injection attacks and defenses in LLMs
seohongpark / ogbench
A benchmark for offline goal-conditioned RL and offline RL
GamePhanesStudio / GamePhanes
An open-source game coding agent environment and benchmark for Godot.
OpenMOSS / EasyWAM
A unified framework for training, fine-tuning, and evaluating World Action Models
arjaythedev / twelve-dollar-server-challenge
Implement one small API on SQLite in any language; we benchmark it on a $12 server (1 vCPU, 2 GB)
open-daimax / daimax-appbench
An automated benchmark platform for AI-generated apps — built-in evaluation rules, standard sample sets, and layered test cases across a multi-dimensional metric system. One pipeline runs build, install, E2E tests, and scoring into a shareable report.
liberomax / LIBERO-MAX
[LIBERO-MAX] A paired benchmark for robot-policy robustness to controlled changes introduced during execution.
aaron-for-value / VeriRun
Evidence-first infrastructure for reproducible, isolated executable evaluation and online rewards.
MirroS-Lab / HarnessEval-W
HarnessEval-W: Agentifying the Evaluation of Visual Worlds
openaiotlab / CUHK-X
[MobiSys 2026] A large-scale, multimodal dataset and benchmark for Human Action Recognition, Understanding and Reasoning
ktwu01 / benchmark-radar
Track 33,707+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.
Chengjun023 / agent-smith
Token 超支？快给史密斯打电话！ Better Call Smith. Adaptive model routing, a glassy Codex usage monitor, and reproducible benchmarks with receipts.
jinzijian / EvoTrace
Compile real-world Claude Code and Codex trajectories into verified, tradable post-training assets.
LynnReal-AI / LynnReal-Omni
LynnReal-Omni brings text-to-video, image-to-video, human- and hand-pose guided generation, structural control, omni-reference generation, style transfer, video editing, degraded-video restoration and streaming long-video generation into a single framework, all at four-step fast generation.
xiaobright / modeltest
Personal LLM engineering-maintenance evaluation harness (V4.1b, frozen). Not a public benchmark.
TriWorldBench / TriWorldBench
A Benchmark Evaluating Triple-View Embodied World Models
UditAkhourii / cdaf
CDAF (Cached Descriptive Asset Files) - open sidecar format for video so AI agents stop re-analyzing the same footage. Spec, CLI, agent skill, reproducible benchmark.
HarlandZZC / OmniVChat
Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue
w1u2d3i4 / multitown
Code-only runtime toolkit for cost-aware multi-agent organization and control
EastForce / worker-ai-spark
开源劳动者AI研究与评测项目：建设劳动知识、公开评测基准与共同治理机制｜Open worker-centered AI research and evaluation project.
aiming-lab / RSI-Exam
RSI-Exam: Measuring Recursive Self-Improvement on Long-Horizon, Executable Research Tasks · Discord https://discord.gg/FBzhepyE
SergiioB / intel-arc-pro-b70-inference-cookbook
Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!
KaiWU5 / Awesome-AI4AI
Updated weekly — until humanity achieves AI4AI. AI4AI Survey: can AI reliably improve AI? 223 papers on long-horizon agents, benchmarks, harness design, and recursive self-improvement.
KhanCold / merchantbench
MerchantBench is a 365-day, order-level benchmark for evaluating the long-term coherence of LLM agents in seller-side e-commerce operations.
solis-team / XRepoTest
[EMNLP 2026] A multilingual benchmark for repository-level unit test generation, evaluating LLMs across 5 programming languages and real-world code under reproducible, Docker-based execution.
kairunwen / Awesome-Robot-Use-Agent
A curated collection of papers, research blogs, open-source tools, benchmarks, and community demos for robot-use agents, including demos powered by GPT-6 Astra. —— Explore our searchable website.
apimart001a / llm-gateway-comparison
LLM gateway comparison: self-hosted and managed AI API gateways compared on protocol compatibility, billing units, timeout and retry semantics, idempotency, quotas and observability.
AMAP-ML / LoopArena
Benchmarking models as runtime Controllers for Loop Engineering
Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
Shuo-Liang-0111 / RA-Bench
Real-event-anchored benchmark for detecting AI-generated videos in real-world crisis settings.
LINs-lab / UltraText_Bench
UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation
tanishq-dubey / macos-laguna-s2.1
Community benchmarks and scripts for running Poolside Laguna S 2.1
lordx64 / pentestkit
Autonomous multi-agent pentest framework — plans, exploits, verifies (proof-required), CVSS-scores and writes client-ready reports. 104/104 (100%) on the XBOW validation benchmarks, powered by Kimi K3.
seohongpark / ocbench
A controllable benchmark for robotic manipulation
ernie-research / FoR-T2I
Official repository of FoR-T2I, a benchmark evaluating T2I models from a different perspective.
YuyaoGe / doc2harness
Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "
Heman10x-NGU / Verdict-open-jev
Non-autoregressive decision engine on ModernBERT (151M) with calibrated uncertainty (RLCD), TypeSafe AI Jev benchmark audit, and in-browser WebGPU playground
ArtificialAnalysis / aa-agentperf-local
Benchmark local LLM serving by replaying real agent trajectories
ruyianry / JevGym
JevGym is an open-source platform designed to benchmark and facilitate better probabilistic estimation in Jev-alike models
InkMind-AI / Dyn-3D
TempoVista is the official implementation of Dyn-3D, a benchmark and training pipeline for 3D spatial reasoning in vision-language models. The release contains frame-selection tools, SFT and Kinematic-GSPO training code, benchmark evaluation scripts, Dyn-3D annotations, and checkpoints for Qwen3-VL and InternVL3.5.
dlgjr / FINAR-VL
Open-source financial multimodal LLM for document understanding, chart/table reasoning, numerical reasoning, and financial analysis, with SFT, dual-branch RL, MOPD distillation, and public benchmark evaluation.
cchen1436 / JarvisBench
Benchmark for allocating scarce user attention during long-horizon agent work
HongyeYangGT / DepthBenchCAD
A three-level benchmark for evaluating audit depth, evidence allocation, and reliability in generative CAD systems.
Simreal-AI / Xitadel-QuantBench
[Public preview] The first public quant trading benchmark scored against what professional traders actually made on the same asset
Simreal-AI / MathmoBench
[Public preview] An advanced AI Benchmark for math, produced by mathmo at St John's College, Cambridge.
Simreal-AI / hard-puzzle-benchmark
[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.
DAOZHENREN / trace2task
Turn successful workflows into resettable, verifiable benchmarks for AI agents.
bpy-dev / blender-mcp
Independent enhanced distribution of Blender Lab MCP: standalone bpy, runtime API documentation, and benchmark tooling
hsj576 / virtual-ai-infra-team
An open-source virtual AI Infra team that discovers, benchmarks, and safely upgrades local LLM inference—with independent quality gates, a stable OpenAI-compatible API, and automatic rollback.