facebookresearch / swe-sweep
How many bugs can LMs find & fix in large codebases?
facebookresearch / swe-sweep
Omni-Scientist / Awesome-AI-Scientist
🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.
scaleapi / rsi-benchmark
RSI Bench is an ongoing effort to evaluate whether AI agents can develop the capabilities required to advance AI R&D.
ZeamoxWang / pdmd
Official Training and Benchmarking Implementation of PDMD: Projected Distribution Matching Distillation for Video Diffusion Models
Weschera / Qwen3.8-27B-oMLX-MTP-Mac
Qwen 3.8 27B at 48-65 tok/s on a Mac Studio M4 Max — measured oMLX native-MTP recipe, same-day A/B vs mlx-vlm+MTP sidecar, raw benchmark JSON included
brycewang-stanford / many-ppt-skills
Every AI slide-deck skill worth knowing, on one page — both the HTML and native-PPTX routes, compared on what actually decides the choice rather than on star count. Bilingual, auto-refreshed.
imitator-game / The-Imitator-Game
Official repo for The Imitator Game: Benchmarking Robot Imitative Ability Beyond Action Prediction
VAIL-UCLA / NavSafe-infty
NavSafe-∞: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments
Gen-Verse / Skill-Entropy-RL
[NeurIPS 2026] Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
intel / gpu-ai-skills
Agent Skills for running, benchmarking, and profiling Hugging Face models on Intel GPUs — PyTorch, vLLM-XPU, and SGLang-XPU
Junjue-Wang / DORA
[NeurIPS 2026 Oral] DORA: Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations
yzhao062 / pyod
A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.
open-compass / opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
Andyyyy64 / whichllm
Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.
EvolvingLMMs-Lab / lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
open-compass / VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
MichaelGrupp / evo
Python package for the evaluation of odometry and SLAM
devMEremenko / XcodeBenchmark
XcodeBenchmark measures the compilation time of a large codebase on iMac, MacBook, and Mac Pro
RUC-NLPIR / FlashRAG
⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)
embeddings-benchmark / mteb
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
llm-as-a-verifier / llm-as-a-verifier
LLM-as-a-Verifier is a general-purpose framework that provides fine-grained feedback for any agent without requiring additional training. It achieves SOTA performance across coding, robotics, and medical agentic benchmarks.
yandex / yandex-tank
Load and performance benchmark tool
elastic / rally
Macrobenchmarking framework for Elasticsearch
GoogleCloudPlatform / PerfKitBenchmarker
PerfKit Benchmarker (PKB) contains a set of benchmarks to measure and compare cloud offerings. The benchmarks use default settings to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.
petergpt / bullshit-benchmark
BullshitBench measures whether AI models challenge nonsensical prompts instead of confidently answering them, created by Peter Gostev.
RoboVerseOrg / RoboVerse
RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning
mlcommons / training
Reference implementations of MLPerf® training benchmarks
opendilab / LightZero
[NeurIPS 2023 Spotlight] LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios (awesome MCTS)
yoshitomo-matsubara / torchdistill
A coding-free framework built on PyTorch for reproducible deep learning studies. PyTorch Ecosystem. 🏆26 knowledge distillation methods presented at TPAMI, CVPR, ICLR, ECCV, NeurIPS, ICCV, AAAI, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
huhusmang / Awesome-LLMs-for-Vulnerability-Detection
The community's most comprehensive, continuously-updated index of research on Large Language Models for software vulnerability detection — papers across function-level, repository-level, agentic, and smart-contract detection, plus datasets, benchmarks, and surveys.
evo-hq / evo
turns your codebase into an autoresearch loop — discovers what to measure, instruments the benchmark, then runs tree search with parallel subagents.
sintel-dev / Orion
Unsupervised time series anomaly detection library
kengz / SLM-Lab
Modular Deep Reinforcement Learning framework in PyTorch. Companion library of the book "Foundations of Deep Reinforcement Learning".
LiveBench / LiveBench
LiveBench: A Challenging, Contamination-Free LLM Benchmark
NVIDIA-NeMo / Gym
Evaluate and improve models and agents using environments
chunxiaoxx / nautilus-compass
Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
zilliztech / VectorDBBench
Benchmark for vector databases.
Mibayy / token-savior
MCP server that gets Claude to 97.9% (188/192) on a real coding benchmark at -80% active tokens and -83% wall time, vs 78.3% plain. Structural code navigation + persistent memory engine. Works with every MCP client.
usamahz / cpu-performance-engineering
A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.
NeuroTechX / moabb
Mother of All BCI Benchmarks
pytorch / benchmark
TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.
redis / memtier_benchmark
NoSQL Redis and Memcache traffic generation and benchmarking tool.
python / pyperformance
Python Performance Benchmark Suite
airspeed-velocity / asv
Airspeed Velocity: A simple Python benchmarking tool with web-based reporting
geerlingguy / sbc-reviews
Jeff Geerling's SBC review data - Raspberry Pi, Radxa, Orange Pi, etc.
psf / pyperf
Toolkit to run Python benchmarks
google-research / android_world
AndroidWorld is an environment and benchmark for autonomous agents
BinWang28 / audio-ai-hub
The hub for audio AI research: papers, open models, benchmarks & datasets across audio LLMs, speech recognition, TTS, music & audio generation.
LAMDA-Tabular / TALENT
A comprehensive toolkit and benchmark for tabular data learning, featuring 35+ deep methods, more than 10 classical methods, and 300 diverse tabular datasets.
Ayanami0730 / deep_research_bench
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
yeyuxin777 / baoyan-agent-benchmark
Sprix AI at 屿智同行 × 保研岛 — an open-source benchmark for AI agents in Baoyan advising and application planning.
Picovoice / speech-to-text-benchmark
speech to text benchmark framework
TIGER-AI-Lab / VLM2Vec
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
layumi / University1652-Baseline
ACM Multimedia2020 University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization :helicopter: annotates 1652 buildings in 72 universities around the world.
KarhouTam / FL-bench
Benchmark of federated learning. Dedicated to the community. 🤗
EverMind-AI / SkillCorpus
Open-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.
Ar9av / PaperOrchestra
An automated AI research-paper writer based off Google's PaperOrchestra paper's implementation through a skills - benchmark + autoraters using any coding agent (Claude Code, Cursor, Antigravity, Cline, Aider). No API keys, no LLM SDKs.
ServiceNow / AgentLab
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
mlcommons / ck
Collective Knowledge (CK), Collective Mind (CM/CMX) and MLPerf automations: community-driven projects to learn how to run AI, ML, and other emerging workloads more efficiently and cost-effectively across diverse models, datasets, software, and hardware using MLPerf methodology and benchmarks.
google-research / weatherbench2
A benchmark for the next generation of data-driven global weather models.