MadsLorentzen / ai-job-search
The job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it.
MadsLorentzen / ai-job-search
mlflow / mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
comet-ml / opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
oumi-ai / oumi
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
interviewstreet / hiring-agent
AI agent to evaluate and score resumes.
Kiln-AI / Kiln
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
transformerlab / transformerlab-app
The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.
rafaelpadilla / Object-Detection-Metrics
Most popular metrics used to evaluate object detection algorithms.
evalstate / fast-agent
Code, Build and Evaluate agents - excellent Model and Skills/MCP/ACP/A2A Support
huggingface / evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
tsale / EDR-Telemetry
This project aims to compare and evaluate the telemetry of various EDR products.
vllm-project / guidellm
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
ray-r-ren / agent-apprenticeship
The living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents.
harveyai / harvey-labs
A benchmark built to evaluate and improve agent capabilities for supporting legal work.
zjunlp / SkillNet
Create, Evaluate, and Connect AI Skills
NVIDIA-NeMo / Gym
Evaluate and improve models and agents using environments
mozilla-ai / any-agent
A single interface to use and evaluate different agent frameworks
EmbeddedLLM / JamAIBase
The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.
pytorch / benchmark
TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.
nolabs-ai / deepfabric
Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline
huggingface / upskill
Generate and evaluate agent skills for code agents like Claude Code, Open Code, OpenAI Codex
huggingface / Repo2RLEnv
Turn any repository into verifiable RL environments for coding agents - Harbor tasks you can train on, evaluate and share on the Hugging Face Hub
MIC-DKFZ / nnDetection
nnDetection is a self-configuring framework for 3D (volumetric) medical object detection which can be applied to new data sets without manual intervention. It includes guides for 12 data sets that were used to develop and evaluate the performance of the proposed method.
allenai / vla-evaluation-harness
One framework to evaluate any VLA model on any robot simulation benchmark.
pymc-devs / pytensor
PyTensor allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays.
NVIDIA / SkillEvaluator
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
aws-samples / service-screener-v2
A tool for customers to evaluate their AWS service configurations based on AWS and community best practices and receive recommendations on potential improvements.
facebookresearch / meta-agents-research-environments
Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
ChidiebereNjoku / ModelBrief
An open-source Python library that automates the analysis and reporting of machine learning experiments, generating structured and comprehensive reports that make model performance, results, and key insights easier to understand and evaluate by data professionals, with optional Groq-powered AI explanations and recommendations for deeper insights.
i3T4AN / KADATH
Evolutionary multi-agent runtime that breeds, evaluates, and improves autonomous agents across reproducible epochs to converge on optimization of a goal.
To3akaRin / VLMForge
Evaluate visual models on your own images, JSON Schema, and production constraints with LangGraph, Pareto analysis, and a no-key Replay demo.
tomsen02 / oss-audit
Evaluate GitHub projects from an outside contributor's perspective. An agent skill with a read-only collector and evidence-based methodology.
scaleapi / rsi-benchmark
RSI Bench is an ongoing effort to evaluate whether AI agents can develop the capabilities required to advance AI R&D.
zjunlp / AutoSciRub
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
HERRY423 / BioNexus
Scientific reliability layer for AI-assisted biology: evaluates what evidence warrants, caps unsupported claims, audits analytical validity, and prevents silent scientific overreach.
datosh / nixos-vex
OpenVEX feed and NixOS module to manage NixOS CVEs, evaluated against your system's configuration.
volt-l18 / llm-redteam-pipeline
An automated, multi-turn LLM red-teaming pipeline that pits an Attacker model against a Target model to evaluate safety boundaries, featuring LLM-as-a-Judge scoring and an interactive HTML vulnerability dashboard.