zjunlp / SkillNet
Create, Evaluate, and Connect AI Skills
zjunlp / SkillNet
mozilla-ai / any-agent
A single interface to use and evaluate different agent frameworks
langchain-ai / openevals
Readymade evaluators for your LLM apps
agentscope-ai / OpenJudge
OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards
AISBench / benchmark
AISBench Benchmark is a model evaluation tool built on OpenCompass, compatible with OpenCompass’s configuration system, dataset structure, and model backend implementation, while extending support for service-based models.
Purewhiter / mobilegym
[EMNLP 2026] MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research · 浏览器里运行的安卓模拟器 · Browser-hosted Android Simulator · Verifiable Evaluation · Scalable Online RL Training
huggingface / Repo2RLEnv
Turn any repository into verifiable RL environments for coding agents - Harbor tasks you can train on, evaluate and share on the Hugging Face Hub
overmind-core / overmind
The platform for continuously improving AI agents.
allenai / vla-evaluation-harness
One framework to evaluate any VLA model on any robot simulation benchmark.
pymc-devs / pytensor
PyTensor allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays.
isaac-sim / IsaacLab-Arena
Isaac Lab - Arena is a robotics simulation framework that enhances NVIDIA Isaac Lab by providing a composable, scalable system for creating diverse simulation environments and evaluating robot learning policies. The framework enables developers to rapidly prototype and test robotic tasks with various robot embodiments, objects, and environments.
NVIDIA / SkillEvaluator
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
rockmatrix-ai / EdgeRock
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
dengmeiluan / ticket-monitoring
✈️ 自托管多渠道机票价格监控：五平台明细采集 → 钉钉图文推送 → 达标四层强提醒。数据留在本机，配置全在网页。Self-hosted flight price monitoring with a dispatch-board console.
whichmen / dxl-commerce-agent
OpenClaw + 大模型驱动的完整电商客服自动化系统：覆盖淘宝、拼多多、抖音、快手、小红书、视频号，包含浏览器/手机 Worker、订单物流与售后工具、记忆、回执、恢复和 Eval。
whitecircle / rise
RISE: Red-teaming via Iterative Strategy Evolution (NeurIPS 2026)
SimpleJev / JevAny
Open infrastructure for training, evaluating, and deploying System 1 decision models across language and multimodal backbones.
scaleapi / rsi-benchmark
RSI Bench is an ongoing effort to evaluate whether AI agents can develop the capabilities required to advance AI R&D.
xiaojunlan / care
CARE: corrective data generation and FSR-Bench evaluation for robotic manipulation
AlayaLab / PROWBench
PROWBench: evaluating whether video models faithfully render what a program specifies.
AlexJ1ang / sse-agent-bench
Configuration-driven evaluation, regression testing, replay, and load testing for SSE-streaming LLM agents
ShishirPatil / gorilla
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
oumi-ai / oumi
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
open-compass / opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
transformerlab / transformerlab-app
The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.
rafaelpadilla / Object-Detection-Metrics
Most popular metrics used to evaluate object detection algorithms.
EvolvingLMMs-Lab / lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
open-compass / VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
MichaelGrupp / evo
Python package for the evaluation of odometry and SLAM
truera / trulens
Evaluation and Tracking for LLM Experiments and AI Agents
embeddings-benchmark / mteb
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
domokane / FinancePy
A Python Finance Library that focuses on the pricing and risk-management of Financial Derivatives, including fixed-income, equity, FX and credit derivatives.
UKGovernmentBEIS / inspect_ai
Inspect: A framework for large language model evaluations
huggingface / lighteval
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
pydata / numexpr
Fast numerical array expression evaluator for Python, NumPy, Pandas, PyTables and more
huggingface / evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
Sahir619 / fable-method
The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.
galilai-group / stable-worldmodel
A platform for reproducible world model research and evaluation
Cloud-CV / EvalAI
:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
pykeen / pykeen
🤖 A Python library for learning and evaluating knowledge graph embeddings
msoedov / agentic_security
Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪
tsale / EDR-Telemetry
This project aims to compare and evaluate the telemetry of various EDR products.
rentruewang / aioway
AI on the way. An auto deep learning pipe dream. An RDBMS approach to deep learning. Declarative, explainable, scalable, optimizable, easy to deploy, all that good stuff.
InterDigitalInc / CompressAI
A PyTorch library and evaluation platform for end-to-end compression research
ray-r-ren / agent-apprenticeship
The living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents.
sepandhaghighi / pycm
Multi-class confusion matrix library in Python
toluaina / pgsync
Postgres, MySQL, or MariaDB to Elasticsearch/OpenSearch sync
tatuylonen / wiktextract
Wiktionary dump file parser and multilingual data extractor
NVIDIA-NeMo / Gym
Evaluate and improve models and agents using environments
chunxiaoxx / nautilus-compass
Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
Mathics3 / mathics-core
An open-source Mathematica Kernel. This repository contains the Python modules for WL Built-in functions, variables, core primitives, e.g. Symbol, a parser to create Expressions, and an evaluator to execute them.
cvs-health / uqlm
[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification
scoder / lupa
Lua in Python
EmbeddedLLM / JamAIBase
The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.
JudgmentLabs / judgeval
The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.
braintrustdata / autoevals
AutoEvals is a tool for quickly and easily evaluating AI model outputs using best practices.
pytorch / benchmark
TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.
microsoft / AIOpsLab
A holistic framework to enable the design, development, and evaluation of autonomous AIOps agents.
Raudaschl / rag-fusion
RAG-Fusion: multi-query generation + Reciprocal Rank Fusion for better retrieval-augmented generation. Includes evaluation harness with NFCorpus/BEIR.
certsocietegenerale / fame
FAME Automates Malware Evaluation