roboflow / trackers
Trackers gives you clean, modular re-implementations of leading multi-object tracking algorithms released under the permissive Apache 2.0 license. You combine them with any detection model you already use.
roboflow / trackers
evalstate / fast-agent
Code, Build and Evaluate agents - excellent Model and Skills/MCP/ACP/A2A Support
morphik-org / morphik-core
Open-source multimodal retrieval engine (Morphik Core)
truera / trulens
Evaluation and Tracking for LLM Experiments and AI Agents
RUC-NLPIR / FlashRAG
⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)
modelscope / evalscope
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
embeddings-benchmark / mteb
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
superlinked / sie
Open-source inference server and production cluster for all the models your agent needs.
openai / human-eval
Code for the paper "Evaluating Large Language Models Trained on Code"
qdrant / fastembed
Fast, Accurate, Lightweight Python library to make State of the Art Embedding
OpenDCAI / DataFlex
[NeurIPS 2026] DataFlex: A Unified Benchmark and Evaluation Platform for Data-Centric Training of Large Language Models
MemMachine / MemMachine
Universal memory layer for AI Agents. It provides scalable, extensible, and interoperable memory storage and retrieval to streamline AI agent state management for next-generation autonomous systems.
UKGovernmentBEIS / inspect_ai
Inspect: A framework for large language model evaluations
swirlai / swirl-search
AI Search & RAG Without Moving Your Data. Get instant answers from your company's knowledge across 100+ apps while keeping data secure. Deploy in minutes, not months.
illuin-tech / colpali
The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.
jgravelle / jcodemunch-mcp
Cut AI token costs 95%+ on code exploration. The leading MCP server for precise, symbol-level GitHub code retrieval via tree-sitter AST. Works with Claude Code, Cursor & any MCP client. 313B+ tokens saved.
Zleap-AI / SAG
A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.
huggingface / lighteval
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
pydata / numexpr
Fast numerical array expression evaluator for Python, NumPy, Pandas, PyTables and more
huggingface / evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
IBM / AssetOpsBench
AssetOpsBench - Industry 4.0: A unified benchmark and framework for building, orchestrating, and evaluating domain-specific AI agents for Industry 4.0 asset operations and maintenance, with 460+ scenarios, 5 specialist agents (IoT, FMSR, TSFM, Work Order,...), and multi-agent orchestration blueprints (MetaAgent, AgentHive) over MCP.
Sahir619 / fable-method
The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.
galilai-group / stable-worldmodel
A platform for reproducible world model research and evaluation
bespokelabsai / nimble
Local typed decisions, contrastive data curation, and model evaluation.
future-agi / future-agi
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
castorini / pyserini
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.
ShenSeanChen / waku-agent
Waku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all in code built to stay legible as it grows.
Cloud-CV / EvalAI
:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
pykeen / pykeen
🤖 A Python library for learning and evaluating knowledge graph embeddings
youngfish42 / Awesome-FL
Comprehensive and timely academic information on federated learning (papers, frameworks, datasets, tutorials, workshops)
msoedov / agentic_security
Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪
tsale / EDR-Telemetry
This project aims to compare and evaluate the telemetry of various EDR products.
SaiAkhil066 / CORTEX-AI-SUPER-RAG
CORTEX RAG is an enterprise retrieval and knowledge assistant that helps teams find accurate answers from company data with citations, permission-aware retrieval, and fast deployment
NVIDIA-AI-Blueprints / video-search-and-summarization
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
Barca0412 / Introduction-to-Quantitative-Finance
AI+金融（量化）：1.多因子股票量化框架开源教程 2.学界和业界的经典资料收录 3.AI + 金融的相关工作，包括LLM, Agent, benchmark(evaluation), etc.
rentruewang / aioway
AI on the way. An auto deep learning pipe dream. An RDBMS approach to deep learning. Declarative, explainable, scalable, optimizable, easy to deploy, all that good stuff.
xhluca / bm25s
Fast BM25 search in Python, powered by Numpy and Numba
Tencent / WeMM-Embedding
WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.
vllm-project / guidellm
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
InterDigitalInc / CompressAI
A PyTorch library and evaluation platform for end-to-end compression research
ray-r-ren / agent-apprenticeship
The living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents.
KruxAI / ragbuilder
A toolkit to create optimal Production-readyRetrieval Augmented Generation(RAG) setup for your data
sepandhaghighi / pycm
Multi-class confusion matrix library in Python
harveyai / harvey-labs
A benchmark built to evaluate and improve agent capabilities for supporting legal work.
Topdu / OpenOCR
OpenOCR: An Open-Source Toolkit for General-OCR Research and Applications, integrates a unified training and evaluation benchmark, commercial-grade OCR and Document Parsing systems, and faithful reproductions of the core implementations from a wide range of academic papers.
zjunlp / SkillNet
Create, Evaluate, and Connect AI Skills
openai / frontier-evals
OpenAI Frontier Evals
IAAR-Shanghai / Awesome-AI-Memory
Awesome AI Memory | LLM Memory | A curated knowledge base on AI memory for LLMs and agents, covering long-term memory, reasoning, retrieval, and memory-native system design. Awesome-AI-Memory 是一个 集中式、持续更新的 AI 记忆知识库，系统性整理了与 大模型记忆（LLM Memory）与智能体记忆（Agent Memory） 相关的前沿研究、工程框架、系统设计、评测基准与真实应用实践。
NVIDIA-NeMo / Gym
Evaluate and improve models and agents using environments
chunxiaoxx / nautilus-compass
Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
mozilla-ai / any-agent
A single interface to use and evaluate different agent frameworks
heymrun / heym
Build agentic systems. Run them with confidence. Orchestrate agents, automate business processes, inspect every execution, and keep humans in control. Deploy Heym on your own infrastructure.
langchain-ai / openevals
Readymade evaluators for your LLM apps
Mathics3 / mathics-core
An open-source Mathematica Kernel. This repository contains the Python modules for WL Built-in functions, variables, core primitives, e.g. Symbol, a parser to create Expressions, and an evaluator to execute them.
cvs-health / uqlm
[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification
superlinear-ai / raglite
🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL
Azure-Samples / chat-with-your-data-solution-accelerator
A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.
haowang02 / codex-candy-eval
Codex 降智测试
jwuthri-cloud / Tracely-ai
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
perseval-BLR / NeuralScreen
DLSS 5 NR on your whole Windows desktop, in real time with Boost, FG, etc. RTX 30/40/50, 12 languages, user presets, recording with audio, one-window mode.