maoyadongsh / siq-agent-security
Research artifacts for provenance-bound agent authorization, signed effect evidence, and reproducible security evaluation.
maoyadongsh / siq-agent-security
YuanKJing / Jev-as-Policy
The highly anticipated open-source repository for JEV as Policy enables one-click setup of the simulation environment. Evaluations of Astra + JEV on benchmarks such as RoboTwin will also be released soon.
Stahl-G / wikiskill
A self-evolving skill framework for evaluable agent tasks, with reproducible multi-domain validation experiments.
RookieZhou0617 / risk-aware-financial-fraud-detection
Multi-source risk-aware graph learning for financial statement fraud detection under strict temporal evaluation.
PyModel / jev-judge-mcp
Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.
AhmadJeddi / Plant-Disease-Classification
A ResNet50-DCBAM deep learning framework for plant disease classification using the PlantVillage dataset with reproducible training and evaluation pipelines.
hotchpotch / jev-reranker
Jev-powered relevance filtering and reranking for RAG in Python.
xixiaouab / ReaLVR-code
ReaLVR: Grounding latent visual reasoning in visual evidence. Training and evaluation code.
stormwther18 / HYSET
Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction
campusx-official / rag-eval-deepeval
Rag evaluation using DeepEval
datamllab / A2E
—
TapnTell-AI / openniw
Open-source AI-assisted NIW/EB1A/O1 petition preparation — from free evaluation to filing package. Not legal advice.
VectorSpaceLab / AgentSWE
Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimization.
Qin-sor / agent-context-lab
A minimal coding-agent harness built to measure what context management actually costs: constraint survival after compaction (8%→83%), default-FAIL verification against false completions, and KV-cache prefix design (21%→90%). Stdlib-only, real deepseek-chat runs
noyce6983981-max / ocr-vlm-local-retrieval
Local-first OCR-VLM multimodal document retrieval and independent evaluation prototype.
zjunlp / AutoSciRub
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
nickname21kmr / gameops-investigator
Evidence-first game operations incident investigation agent with read-only MCP tools and reproducible evaluations.
UniPat-AI / PaperBenchX
A benchmark for scientific research agents, evaluating end-to-end paper reproduction across 12 research areas.
inworld-ai / open-tts-eval
Open, reproducible TTS/STT evaluation toolkit: WER/CER via Whisper ASR, NISQAv2, audio health, and prosody metrics with offline black-and-white reports.
hastimal / gemini-healthcare-agentic-platform
Beyond RAG: An open-source agentic healthcare search framework using Gemini and Google ADK for query fan-out, evidence retrieval, grounding, citation evaluation, and trustworthy healthcare information discovery.
Simo-Dax / claude-code-creative-statics
Static ads con l'AI: concept, routing Higgsfield vs CSS deterministico, eval e self-learning per Claude Code
theaiautomators / jev-arena
A local evaluation lab for decision models. Compare accuracy, speed and memory requirements through an interactive dashboard, inspectable test cases, workflow replays and shareable reports.
MnemosyneOS / mnemosyne
Zero-dependency local-first AI memory engine: graph memory, multimodal ingestion, reranking, temporal reasoning, hash-chained ledger, capsule compression, memory formation + runtime layers. 52 MCP tools (41 native + 11 compatible). MIT.
m2dumpling / pawbot
A self-hosted AI agent runtime with WebUI, MCP tools, context governance, and Record & Replay for deterministic execution & evaluation.
talhaanwarch / awesome-llmops-platforms
Curated list of open-source LLMOps tools: tracing, evals, AI gateways, guardrails and prompt management. Sorted by stars, refreshed weekly.
RinDig / cost-of-remembering
The Cost of Remembering: filesystem memory matches long-context accuracy on LongMemEval while reading 97% fewer tokens and costing 95% less. Harness, run data, 129 agent-built memories, and paper source.
hotchpotch / bekko-system-one
Small System One decision models (17M–400M) for Yes/No, Choice, Score, and reranking. Training toolkit, Python inference, and ONNX support for Node.js and browsers.
stanford-iris-lab / ScholarCatalyst
A Benchmark for Retrieving Papers that Inspire New Research
realHarryX / KBMR
[ACM MM 2026] Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering
keenableai / needle
NEEDLE. A live search and retrieval benchmark from Keenable.ai
tawdesangeeta1973-coder / ptcg-belief-mcts
Information-Set MCTS-inspired agent and reference simulation for the Pokémon TCG AI Battle Challenge — belief-based decision policy, deck architecture, and evaluation harness for the Strategy Category submission.
ydflow / research-trail
研迹 ResearchTrail｜本地 AI Native投资研究工作台。串联证券、组合、证据报告、论点复审与 Today 简报，让研究有据、判断留痕。Python 业务内核 · Electron 桌面 · 可复现 Agent 评测。
Gen-Verse / PAST-Bench
PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents
jaz-lang / jaz-evals
Reproduction package for "Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity" — ten agent arms across AppWorld and StuLife
hawaiicoco / slotvox
End-to-end spoken language understanding toolkit: a small trainable neural audio codec with RVQ, joint intent-and-slot models over speech features, streaming partial intents, a speech-LLM instruction data pipeline, and turn-level evaluation with reproducible reports. NumPy core with a CPU-only torch extra; synthetic data only, fully offline.
WenjieShu / SolveEdit
SolveEdit-Plan and evaluation code for SolveEdit
gwendolynashe / llm-evaluation-workbench
Argolink-focused LLM evaluation and regression workflow for GPT-6 Sol.
ArcInstitute / cell-eval2
GPU-accelerated evaluation of perturbation prediction models
HongyeYangGT / Part_HackBench
A stress-testing benchmark for partial-completion evaluators in stateful multi-turn agents, focusing on temporary progress, rollback, and attribution failures.
kakuibeyond / talent-eval-agents
—
zsxm1998 / awesome-perception-aware-rlvr
Unified reproductions, benchmarking & evaluation for perception-aware RLVR and OPD in VLMs. 19 methods · 30+ benchmarks · Official code of CGPO (ACM MM 2026 Oral).
Grit-77 / done-is-a-claim
Your coding agent says done. Show the receipt. Evidence-first workflows, practical skills, and plugins for Claude Code and Codex.
PrentisAI / AutoResearchEval
—
gongsun947-jpg / pm-knowledge-qa
Local AI Personal Knowledge Base Q&A · Full‑Link RAG Practical Implementation (Hybrid Retrieval + Rerank Fine‑Ranking + Citation‑Enabled Generation)
augustinepemberton9 / rag-model-integration
Argolink-focused retrieval workflow and model integration for GPT-6 Sol.
ZhuWenjie98 / MMOOC
MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
clementbouni-arch / hilton-yaounde-rag-concierge
A Retrieval-Augmented Generation assistant for hotel guest support — case study on the Hilton Yaounde, built with Hugging Face Transformers, sentence-transformers and pypdf
Human-Agent-Society / reef-eval
🌊 Evaluation infrastructure for self-evolving agents, on the Harbor task standard.
HERRY423 / BioNexus
Scientific reliability layer for AI-assisted biology: evaluates what evidence warrants, caps unsupported claims, audits analytical validity, and prevents silent scientific overreach.
LGAI-Research / PCBWorld
A reinforcement learning environment for PCB autorouting, built on KiCad's push-and-shove router. Includes a decoder-only PPO/GRPO agent, LLM tool-calling and rule-based baselines, and a uniform 3-stage evaluation pipeline. The routing engine ships separately as the PCBWorld-Engine submodule (GPLv3).
slyyr / yappr
A modular local AI agent built with Python and Ollama, featuring tool calling, memory, web search, document processing, embeddings, and RAG.
datosh / nixos-vex
OpenVEX feed and NixOS module to manage NixOS CVEs, evaluated against your system's configuration.
luanmorenommaciel / brief-spec
Type-aware, evidence-backed delivery contract for AI coding harnesses. Different agents in. One predictable human handoff out.
PillCrew / deedchain
Benchmark of report fidelity for browser agents: does the agent's final report tell the truth about what it actually did?
gulucaptain / MiniMax-H3-Reason
[Evaluation] Can Minimax-H3 Reason About the Physical World?
YellowPancake / quant-autoresearch
A minimal framework for Agents to research quantitative strategies under a fixed evaluation protocol.
iam-epiphany / resume-agent
面向"面试官 × 简历主人公"场景的个人简历 RAG 问答 Agent：把简历、证书、荣誉、项目介绍放进知识库，面试官可对简历任意提问（自我介绍、项目深挖、技术八股、HR 素质、简历细节），系统以第一人称自然作答。核心设计是"宽松推理 + 诚实标注"——硬事实（数字/日期/证书名）必须来自检索，证据不足时强制标注"根据现有知识库推测"而非编造。支持多轮追问记忆、检索兜底链、置信度分级回答、访问码闸门与 IP 限流防刷。
baoerger / globex
Cross-border commerce Agent prototype with retrieval, order tools, persistent memory, and observable execution
Zaious / jev-capability-atlas
Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙語 repo。
sumleo / RLCDAlignBench
RLCDAlignBench: 44 alignment-failure detection benchmarks and code for 'Just Ask Jev' (RLCD zero-shot detector of AI alignment failures)