Blackwellboy / model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
MrZoyo / deslop-GPT
Deletion-first Agent Skill for removing test bloat, verification theater, and speculative fallbacks while preserving behavior.
solis-team / CrossCoder
[EMNLP 2026] Beyond Repository Boundaries: Cross-Repository Graph Retrieval for Code Generation
salikhussain71-code / PAKGOV-RAG-project
Research project on bilingual Urdu-English retrieval-augmented generation for Pakistani government and legal documents, with OCR, hybrid retrieval, citation grounding, and evaluation.
fengzhizi715 / YoloWebAgentStarter
Local-first YOLO workbench for annotation, training, evaluation, and AI assistance.
oce-ai / oce
OpenContextEngine is a self-hosted, ACE-compatible code retrieval service. It indexes source files with cAST-aware chunking, stores metadata in PostgreSQL or SQLite, performs dense vector retrieval in Milvus 3.0, and reranks results with an LLM before coverage-aware selection.
LINs-lab / UltraText_Bench
UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation
daimon3332 / Outlook-Oauth-GetToken
A standalone OAuth2 refresh_token retrieval tool for existing Outlook / Hotmail accounts. 面向已有 Outlook / Hotmail 账号的独立 OAuth2 refresh_token 获取工具
ernie-research / FoR-T2I
Official repository of FoR-T2I, a benchmark evaluating T2I models from a different perspective.
deepmemteam / deepmem
Drop-in AI memory layer with 2x faster retrieval and 10x lower cost. Fully compatible with Mem0 API. Migrate in 5 minutes without any code changes. Self-host for free.
Sherlockwz / AMA
ACL 2026 Code and Skills for “AMA: Adaptive Memory via Multi-Agent Collaboration”.
ConstantineChenn / SakikoMind
Enterprise AI customer-service Agent platform featuring hybrid intent recognition, multi-agent orchestration, MCP-powered RAG, dynamic Skills, memory, human handoff, observability and evaluation. 小祥智能客服系统，支持可追溯、可观测与云端部署。
To3akaRin / VLMForge
Evaluate visual models on your own images, JSON Schema, and production constraints with LangGraph, Pareto analysis, and a no-key Replay demo.
danielgshea / jev-as-a-judge
Using Jev as an evaluator.
qzydustin / qforge
A platform for finding, testing, reducing, and evaluating compiler failures across the quantum software stack.
malevrigns / atlas-agent-control-plane
AtlasAgent - an auditable AI agent control plane: evidence-backed memory, governed tool runtime, checkpoint DAG recovery, and a 55-chapter engineering tutorial. FastAPI / Next.js PWA / Textual TUI
123Cx330Yrx / riftcoach-agent
A quality-gated League of Legends post-game coaching agent built on deterministic match data, RAG, and LLM evaluation.
openlayer-ai / jevals
Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.
InkMind-AI / Dyn-3D
TempoVista is the official implementation of Dyn-3D, a benchmark and training pipeline for 3D spatial reasoning in vision-language models. The release contains frame-selection tools, SFT and Kinematic-GSPO training code, benchmark evaluation scripts, Dyn-3D annotations, and checkpoints for Qwen3-VL and InternVL3.5.
dlgjr / FINAR-VL
Open-source financial multimodal LLM for document understanding, chart/table reasoning, numerical reasoning, and financial analysis, with SFT, dual-branch RL, MOPD distillation, and public benchmark evaluation.
HongyeYangGT / DepthBenchCAD
A three-level benchmark for evaluating audit depth, evidence allocation, and reliability in generative CAD systems.
Simreal-AI / Xitadel-QuantBench
[Public preview] The first public quant trading benchmark scored against what professional traders actually made on the same asset
Simreal-AI / MathmoBench
[Public preview] An advanced AI Benchmark for math, produced by mathmo at St John's College, Cambridge.
kkjiangk / agent-py
Personal AIOps engineering project with durable workflows, scoped hybrid retrieval, reliable event delivery, and verified API/browser contracts.
ABCLDZ / open-world-research-reasoning-eval
Public reproducibility bundle for open-world deep-research reasoning evaluation
IvenKooLab / loci
Shared memory for AI coding tools: hybrid search, cited Q&A, harness memory discovery and a built-in Web console. Local index, MCP + HTTP.
Simreal-AI / hard-puzzle-benchmark
[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.
thiientv / godmode
Production-grade Agent Skills for AI coding agents—composable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, and evals.
tomsen02 / oss-audit
Evaluate GitHub projects from an outside contributor's perspective. An agent skill with a read-only collector and evidence-based methodology.
sun461941-hub / ai-project-copilot
🚀 Turn ideas and repositories into showcase-ready AI projects with agents, RAG, multimodal AI, local models, evals, safety checks, and polished demos.
SalesforceAIResearch / Beagle
:dog: Beagle is designed to be the unified platform for the broader Recursive Self-Improvement (RSI) ecosystem. Think of it as Hugging Face Trainer for agent evolution. Its efficient rollout engine enables evaluation and evolution at scale across different agents, datasets, benchmarks, and evolution methods.
MaxwellCCC / hangzhou-mahjong
Hangzhou Mahjong training and evaluation framework with public-information Transformers, paired branch reinforcement learning, and the r2600 checkpoint.
NocodeMrLi / mini-program-engineering-skill-suite
Mini Program Engineering Skill Suite is an Agent Skill suite for evidence‑first mini‑program development. It helps agents bring WeChat or other mini‑program projects from vague intent to reliable engineering work: project intake, product specification, architecture, implementation, UI & device adaptation, debugging, verification, release readiness.
tttboy123 / jlens-evolve
JLens Evolve v3: evidence-centric evolution platform (autonomous Teacher + local Qwen + official SWE-bench native evaluation). Branch codex/v3-evolution-10000 carries transport resilience, Router-key normalization, per-round temperature beam, and baseline fail-soft.
harish-gaggar / mcp-server-gateway
MCP gateway & OAuth MCP servers. LangGraph agent: memory, eval/tracking, PII guard, context trimming. Docker quick start for enterprise agents.
sangelege / forestfix
Verifiable multi-agent patch evaluation infrastructure
ai-evals-course / cartwheel-homeworks
Homework assignments for Evaluating and Improving AI Agents
OpenMOSS / SWE-bench-Science
A benchmark for testing whether coding agents can resolve engineering tasks in scientific software
WaseemGhanem98 / AgentCheck
Behavioral testing for AI agents.
beskvladimir-create / tsiolkovsky-papers
Machine-readable catalogue of fond 555 (K.E. Tsiolkovsky), Archive of the Russian Academy of Sciences: all 2,019 files, 51,008 scans, plus the retrieval pipeline.
ZJU-REAL / ageval
Agent eval on one running base. Swap the agent under test with plugins; run the same dataset anywhere.
Jiarui-0410 / multimodal-product-discovery
Full-stack multimodal product retrieval and personalized reranking with CLIP, FAISS, Spring Boot, FastAPI, and PostgreSQL.
RenaGao / jev-dataops
An open-source JEV-powered workbench for streaming data selection, quality evaluation, automatic LoRA training and held-out model evaluation.
mallahyari / nemotron-asr-streaming-farsi
Streaming Persian (Farsi) speech recognition: fine-tuning NVIDIA Nemotron 3.5 ASR streaming. Data prep, training, evaluation, inference.
Git-on-my-level / self-improving-agent-ecosystem
Reference architecture, contracts, skills, and fixtures for safe evaluator-driven self-improving ecosystems.
AI-Safety-Institute / transect
Transcript analysis for long-horizon agent evaluations
Kiln-AI / Seahaven
A framework for building synthetic worlds for AI agents. Useful for RL and evaluation environments.
neuronto / agentic-resource-discovery
Neuronto Agentic Resource Discovery (ARD) Index. Federated search across every public ARD registry, plus a verified tool index read from each MCP server's own tools/list, hybrid lexical and semantic retrieval, and ARD-Bench.
nahid-sparktales / agent-dispatcher
Repository retrieval and context engine for Claude Code and Codex: finds relevant code, reuses project knowledge, and pairs each task with specialist skills and verification.
david-g-3654 / homebench
Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.
ai-infra-bench / ai-infra-bench
Evaluating frontier models on real-world ai-infra engineering tasks
jack66g / spatial-memory-engine
A next-gen long-term memory engine for AI apps: self-organizing spatial regions, memory dynamics (Ebbinghaus reinforcement / decay / consolidation), and explainable 8-signal retrieval. Runs offline with zero config; works with any OpenAI-compatible LLM/embedding, cloud or local. Chinese-friendly. Benchmarked head-to-head vs Mem0 and bare RAG
wanqili857-byte / fictionforge
多智能体小说写作框架 | Multi-agent novel-writing framework — 每个角色都有自己的脑子,每一章都有质量门禁
arnsri33 / embedflow
Zero downtime embedding upgrades
Kerneta / daidocs
Open plain-text file format for AI memory. Your assistant's long-term memory as .dai files on your disk: readable by Claude, GPT, Gemini, Cursor, local models and grep (all LLM models work). MCP server + hooks for Claude Code, Claude Desktop, Cursor, Windsurf, Codex. 83% LongMemEval-S (GPT-4o), 92% (Claude Fable 5), 10x fewer tokens.
BROccoLi-921 / AI-testing-evaluation
—
susyimes / LAIN-agreement
铃音的记忆写入协议 - an informatics protocol: fragmented, cue-based, symbolic, highly compressed information for AI agents; usable for memory writing and transmission, decodable approximately or via retrieval.
WalrusQuant / sports-analytic-skills
Multi-sport data science Agent Skills library. Portable judgment for sports modeling — doctrine, validation, leakage control, and honest evaluation.
bodepudimuneendra-netizen / laya-jev-GraphRAG
A database-agnostic Agentic GraphRAG framework using swappable System One models (local Laya / cloud Jev). A plug-and-play intelligence layer featuring a complete 4-phase pipeline, continuous evaluation and custom A* traversal for any graph database.
pipelinelabai / paperbench-expanded
PaperBench High-Difficulty and Expanded Tasks: paper reproduction beyond machine learning. 93 tasks in 12 research areas, run in domain-native software and graded by clean replay. 12 public tasks.