MadsLorentzen / ai-job-search
The job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it.
MadsLorentzen / ai-job-search
langfuse / langfuse
🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
mlflow / mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
comet-ml / opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
oumi-ai / oumi
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
evidentlyai / evidently
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
interviewstreet / hiring-agent
AI agent to evaluate and score resumes.
Helicone / helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
Kiln-AI / Kiln
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
transformerlab / transformerlab-app
The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.
rafaelpadilla / Object-Detection-Metrics
Most popular metrics used to evaluate object detection algorithms.
portfolio-performance / portfolio
Track and evaluate the performance of your investment portfolio across stocks, cryptocurrencies, and other assets.
evalstate / fast-agent
Code, Build and Evaluate agents - excellent Model and Skills/MCP/ACP/A2A Support
rpamis / comet
Comet: agent skill harness for turning ideas into evaluated workflows
openlit / openlit
OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.
symfony / expression-language
Provides an engine that can compile and evaluate expressions
Hexagon / croner
Trigger functions or evaluate cron expressions in JavaScript or TypeScript. No dependencies. Most features. Node. Deno. Bun. Browser.
huggingface / evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
deer-flow / llm-space
A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.
tsale / EDR-Telemetry
This project aims to compare and evaluate the telemetry of various EDR products.
benchflow-ai / skillsbench
SkillsBench evaluates how well skills work and how effective agents are at using them.
vllm-project / guidellm
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
ray-r-ren / agent-apprenticeship
The living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents.
harveyai / harvey-labs
A benchmark built to evaluate and improve agent capabilities for supporting legal work.
zjunlp / SkillNet
Create, Evaluate, and Connect AI Skills
silverhack / monkey365
Monkey365 is an open-source security assessment tool for Microsoft 365, Azure, and Microsoft Entra ID. It helps security professionals identify misconfigurations, review cloud security posture, and evaluate environments against industry security best practices and compliance standards.
microsoft / prompty
Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.
NVIDIA-NeMo / Gym
Evaluate and improve models and agents using environments
mozilla-ai / any-agent
A single interface to use and evaluate different agent frameworks
leiweibau / Pi.Alert
Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns if a "always connected" device disconnects. In addition, it is possible to check web services for availability. For this purpose HTTP status codes and the response time of the service are evaluated.
sunblaze-ucb / exploitgym
ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities designed to evaluate AI agents' ability to develop exploits.
ezylang / EvalEx
EvalEx is a handy expression evaluator for Java, that allows to evaluate simple mathematical and boolean expressions.
EmbeddedLLM / JamAIBase
The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.
pytorch / benchmark
TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.
rubysolo / dentaku
math and logic formula parser and evaluator
nrepl / nrepl
A Clojure network REPL that provides a server and client, along with some common APIs of use to IDEs and other tools that may need to evaluate Clojure code in remote environments.
nolabs-ai / deepfabric
Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline
huggingface / upskill
Generate and evaluate agent skills for code agents like Claude Code, Open Code, OpenAI Codex
huggingface / Repo2RLEnv
Turn any repository into verifiable RL environments for coding agents - Harbor tasks you can train on, evaluate and share on the Hugging Face Hub
google / yggdrasil-decision-forests
A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.
MIC-DKFZ / nnDetection
nnDetection is a self-configuring framework for 3D (volumetric) medical object detection which can be applied to new data sets without manual intervention. It includes guides for 12 data sets that were used to develop and evaluate the performance of the proposed method.
allenai / vla-evaluation-harness
One framework to evaluate any VLA model on any robot simulation benchmark.
pymc-devs / pytensor
PyTensor allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays.
NVIDIA / SkillEvaluator
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
aws-samples / service-screener-v2
A tool for customers to evaluate their AWS service configurations based on AWS and community best practices and receive recommendations on potential improvements.
facebookresearch / meta-agents-research-environments
Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
quick-perf / quickperf
QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties
microsoft / ghqr
GitHub Quick Review: Evaluate your enterprise and organizations against GitHub best practices
itsmostafa / system-one-connector
System One MCP connector to evaluate anything fast and cheap. Give your AI agent direct access to models like: Jev, D1, CLM and Laya
ChidiebereNjoku / ModelBrief
An open-source Python library that automates the analysis and reporting of machine learning experiments, generating structured and comprehensive reports that make model performance, results, and key insights easier to understand and evaluate by data professionals, with optional Groq-powered AI explanations and recommendations for deeper insights.
i3T4AN / KADATH
Evolutionary multi-agent runtime that breeds, evaluates, and improves autonomous agents across reproducible epochs to converge on optimization of a goal.
To3akaRin / VLMForge
Evaluate visual models on your own images, JSON Schema, and production constraints with LangGraph, Pareto analysis, and a no-key Replay demo.
AgentX-ai / AgentX-Trace-Eval
The trust layer for your AI agent on any platform. Trace and evaluate your agent. Full Observability. Run Model portability analysis. Autotune your agent.
tomsen02 / oss-audit
Evaluate GitHub projects from an outside contributor's perspective. An agent skill with a read-only collector and evidence-based methodology.
scaleapi / rsi-benchmark
RSI Bench is an ongoing effort to evaluate whether AI agents can develop the capabilities required to advance AI R&D.
ustayready / Thunderstorm
A collector and derivation engine. It maps your environment, evaluates effective permissions and trust, and writes a complete attack graph as a single .rage.ndjson file. The input to everything else.
shahidazam2020-oss / Student-Awareness-And-Acceptance-Of-Block-Chain-Authentication-framework-In-IoT-Applications
A quantitative research study on blockchain-based authentication for IoT security, examining student awareness, perceived ease of use, security benefits, usefulness, and acceptance. The study applies reliability analysis, confirmatory factor analysis, correlation, and mediation analysis to evaluate key relationships.
RNetworks2019 / fine-tuning-playground
Mastering LLM Fine-Tuning in 2026: A Practical Guide from Hugging Face Basics to Production
STEAM-DROPS / finetune-lab-notes
Fine-Tuning Transformers in 2026: Hands-On LLM Training, LoRA, and PEFT Guide
ryuxdsgn / design-intelligence
RYUX gives AI agents the design intelligence to understand product context, make intentional design decisions, and evaluate what they create.