danielgshea / jev-as-a-judge
Using Jev as an evaluator.
danielgshea / jev-as-a-judge
ChetasLua / jevmeter
Put a live Jev (TypeSafe) meter on any video: every sentence scored, rendered as a 16:9 edit
qzydustin / qforge
A platform for finding, testing, reducing, and evaluating compiler failures across the quantum software stack.
123Cx330Yrx / riftcoach-agent
A quality-gated League of Legends post-game coaching agent built on deterministic match data, RAG, and LLM evaluation.
openlayer-ai / jevals
Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.
InkMind-AI / Dyn-3D
TempoVista is the official implementation of Dyn-3D, a benchmark and training pipeline for 3D spatial reasoning in vision-language models. The release contains frame-selection tools, SFT and Kinematic-GSPO training code, benchmark evaluation scripts, Dyn-3D annotations, and checkpoints for Qwen3-VL and InternVL3.5.
dlgjr / FINAR-VL
Open-source financial multimodal LLM for document understanding, chart/table reasoning, numerical reasoning, and financial analysis, with SFT, dual-branch RL, MOPD distillation, and public benchmark evaluation.
HongyeYangGT / DepthBenchCAD
A three-level benchmark for evaluating audit depth, evidence allocation, and reliability in generative CAD systems.
xGoldy / lucidcv
ATS-safe LaTeX template for modern resumes and CVs. LuaLaTeX, real word spaces, tagged structure, PDF/A-2b, verified against 13 text extractors.
Simreal-AI / Xitadel-QuantBench
[Public preview] The first public quant trading benchmark scored against what professional traders actually made on the same asset
Simreal-AI / MathmoBench
[Public preview] An advanced AI Benchmark for math, produced by mathmo at St John's College, Cambridge.
HeCC928 / shanghai-news-insight
Interactive Shanghai A-share research terminal with DeepSeek agents, evidence-linked news analysis, and a reproducible DCF valuation engine. Built with Next.js, FastAPI, and Python.
ABCLDZ / open-world-research-reasoning-eval
Public reproducibility bundle for open-world deep-research reasoning evaluation
Simreal-AI / hard-puzzle-benchmark
[Public preview] Hard human-written reasoning puzzles: 749 answered items in 15 topics (707 scored core), with protocol and scorer.
thiientv / godmode
Production-grade Agent Skills for AI coding agentsβcomposable workflows for planning, TDD, debugging, review, UI/UX, releases, incidents, and evals.
tomsen02 / oss-audit
Evaluate GitHub projects from an outside contributor's perspective. An agent skill with a read-only collector and evidence-based methodology.
SalesforceAIResearch / Beagle
:dog: Beagle is designed to be the unified platform for the broader Recursive Self-Improvement (RSI) ecosystem. Think of it as Hugging Face Trainer for agent evolution. Its efficient rollout engine enables evaluation and evolution at scale across different agents, datasets, benchmarks, and evolution methods.
MaxwellCCC / hangzhou-mahjong
Hangzhou Mahjong training and evaluation framework with public-information Transformers, paired branch reinforcement learning, and the r2600 checkpoint.
NocodeMrLi / mini-program-engineering-skill-suite
Mini Program Engineering Skill Suite is an Agent Skill suite for evidenceβfirst miniβprogram development. It helps agents bring WeChat or other miniβprogram projects from vague intent to reliable engineering work: project intake, product specification, architecture, implementation, UI & device adaptation, debugging, verification, release readiness.
tttboy123 / jlens-evolve
JLens Evolve v3: evidence-centric evolution platform (autonomous Teacher + local Qwen + official SWE-bench native evaluation). Branch codex/v3-evolution-10000 carries transport resilience, Router-key normalization, per-round temperature beam, and baseline fail-soft.
harish-gaggar / mcp-server-gateway
MCP gateway & OAuth MCP servers. LangGraph agent: memory, eval/tracking, PII guard, context trimming. Docker quick start for enterprise agents.
sangelege / forestfix
Verifiable multi-agent patch evaluation infrastructure
ai-evals-course / cartwheel-homeworks
Homework assignments for Evaluating and Improving AI Agents
OpenMOSS / SWE-bench-Science
A benchmark for testing whether coding agents can resolve engineering tasks in scientific software
Janina21 / Crypto-Checker
Python-based crypto wallet analysis tool with multi-threaded address and seed phrase checking (up to 50 threads). Supports Bitcoin, Ethereum, Solana, BNB Chain, Polygon, and EVM networks. Includes proxy rotation, portfolio aggregation, USD valuation, token scanning, and TXT/CSV/JSON export for auditing, research, and analytics.
WaseemGhanem98 / AgentCheck
Behavioral testing for AI agents.
ZJU-REAL / ageval
Agent eval on one running base. Swap the agent under test with plugins; run the same dataset anywhere.
RenaGao / jev-dataops
An open-source JEV-powered workbench for streaming data selection, quality evaluation, automatic LoRA training and held-out model evaluation.
mallahyari / nemotron-asr-streaming-farsi
Streaming Persian (Farsi) speech recognition: fine-tuning NVIDIA Nemotron 3.5 ASR streaming. Data prep, training, evaluation, inference.
Git-on-my-level / self-improving-agent-ecosystem
Reference architecture, contracts, skills, and fixtures for safe evaluator-driven self-improving ecosystems.
AI-Safety-Institute / transect
Transcript analysis for long-horizon agent evaluations
Kiln-AI / Seahaven
A framework for building synthetic worlds for AI agents. Useful for RL and evaluation environments.
david-g-3654 / homebench
Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.
ai-infra-bench / ai-infra-bench
Evaluating frontier models on real-world ai-infra engineering tasks
wanqili857-byte / fictionforge
ε€ζΊθ½δ½ε°θ―΄εδ½ζ‘ζΆ | Multi-agent novel-writing framework β ζ―δΈͺθ§θ²ι½ζθͺε·±ηθε,ζ―δΈη« ι½ζ质ιι¨η¦
BROccoLi-921 / AI-testing-evaluation
β
WalrusQuant / sports-analytic-skills
Multi-sport data science Agent Skills library. Portable judgment for sports modeling β doctrine, validation, leakage control, and honest evaluation.
bodepudimuneendra-netizen / laya-jev-GraphRAG
A database-agnostic Agentic GraphRAG framework using swappable System One models (local Laya / cloud Jev). A plug-and-play intelligence layer featuring a complete 4-phase pipeline, continuous evaluation and custom A* traversal for any graph database.
binxgtl / luau-vmp-deobf
SUPPORT MoonVeil v1.4.5, Luraph v14.7
pipelinelabai / paperbench-expanded
PaperBench High-Difficulty and Expanded Tasks: paper reproduction beyond machine learning. 93 tasks in 12 research areas, run in domain-native software and graded by clean replay. 12 public tasks.
maoyadongsh / siq-agent-security
Research artifacts for provenance-bound agent authorization, signed effect evidence, and reproducible security evaluation.
YuanKJing / Jev-as-Policy
The highly anticipated open-source repository for JEV as Policy enables one-click setup of the simulation environment. Evaluations of Astra + JEV on benchmarks such as RoboTwin will also be released soon.
Stahl-G / wikiskill
A self-evolving skill framework for evaluable agent tasks, with reproducible multi-domain validation experiments.
RookieZhou0617 / risk-aware-financial-fraud-detection
Multi-source risk-aware graph learning for financial statement fraud detection under strict temporal evaluation.
PyModel / jev-judge-mcp
Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.
AhmadJeddi / Plant-Disease-Classification
A ResNet50-DCBAM deep learning framework for plant disease classification using the PlantVillage dataset with reproducible training and evaluation pipelines.
xixiaouab / ReaLVR-code
ReaLVR: Grounding latent visual reasoning in visual evidence. Training and evaluation code.
campusx-official / rag-eval-deepeval
Rag evaluation using DeepEval
datamllab / A2E
β
TapnTell-AI / openniw
Open-source AI-assisted NIW/EB1A/O1 petition preparation β from free evaluation to filing package. Not legal advice.
MrChengl11 / hd2-lua-mod-skill
a skill make agent know how to build a lua mod of helldivers 2
VectorSpaceLab / AgentSWE
Can coding agents build the agent you actually want? A benchmark of 25 agent software engineering tasks across creation, editing and optimization.
Qin-sor / agent-context-lab
A minimal coding-agent harness built to measure what context management actually costs: constraint survival after compaction (8%β83%), default-FAIL verification against false completions, and KV-cache prefix design (21%β90%). Stdlib-only, real deepseek-chat runs
noyce6983981-max / ocr-vlm-local-retrieval
Local-first OCR-VLM multimodal document retrieval and independent evaluation prototype.
zjunlp / AutoSciRub
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
nickname21kmr / gameops-investigator
Evidence-first game operations incident investigation agent with read-only MCP tools and reproducible evaluations.
UniPat-AI / PaperBenchX
A benchmark for scientific research agents, evaluating end-to-end paper reproduction across 12 research areas.
inworld-ai / open-tts-eval
Open, reproducible TTS/STT evaluation toolkit: WER/CER via Whisper ASR, NISQAv2, audio health, and prosody metrics with offline black-and-white reports.
hastimal / gemini-healthcare-agentic-platform
Beyond RAG: An open-source agentic healthcare search framework using Gemini and Google ADK for query fan-out, evidence retrieval, grounding, citation evaluation, and trustworthy healthcare information discovery.
theaiautomators / jev-arena
A local evaluation lab for decision models. Compare accuracy, speed and memory requirements through an interactive dashboard, inspectable test cases, workflow replays and shareable reports.