sumuru789 / DK-VLM
Official implementation for "DK-VLM: Geometry as Position for Spatial Reasoning in Vision-Language Models"
sumuru789 / DK-VLM
huggingface / transformers
๐ค Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
hiyouga / LlamaFactory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
sgl-project / sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
StarTrail-org / PixelRAG
https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
OpenRLHF / OpenRLHF
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
oumi-ai / oumi
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
jingyaogong / minimind-v
๐ Train a 65M-parameter VLM from scratch in just 2h!
OpenBMB / UltraRAG
A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines
Blaizzy / mlx-vlm
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
xlite-dev / Awesome-LLM-Inference
๐A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.๐
EvolvingLMMs-Lab / lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
open-compass / VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
huggingface / smollm
Everything about the SmolLM and SmolVLM family of models
PaddlePaddle / FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
modelscope / evalscope
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
xlang-ai / OSWorld
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
radixark / miles
Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.
lightseekorg / tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
om-ai-lab / OmAgent
[EMNLP-2024] Build multimodal language agents for fast prototype and production
zai-org / GLM-V
GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
Tencent-Hunyuan / HunyuanOCR
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
NVIDIA-AI-Blueprints / video-search-and-summarization
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
om-ai-lab / VLX-Seek
VLX-Seek is an efficient VLM that grounds fine-grained visual targets with high precision in wild environments.
Tencent / AngelSlim
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
intel / auto-round
PTQ-Level Efficiency, QAT-Level Accuracy|่ฎญ็ปๅ้ๅ็ๆ็๏ผ้ๅๆ็ฅ่ฎญ็ป็็ฒพๅบฆ
wenet-e2e / wespeaker
Research and Production Oriented Speaker Verification, Recognition and Diarization Toolkit
zai-org / CogAgent
An open-sourced end-to-end VLM-based GUI Agent
NVIDIA-NeMo / Automodel
๐ Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
jjang-ai / vmlx
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
OpenDCAI / OpenWorldLib
Unified Codebase for Advanced World Models.
Purewhiter / mobilegym
[EMNLP 2026] MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research ยท ๆต่งๅจ้่ฟ่ก็ๅฎๅๆจกๆๅจ ยท Browser-hosted Android Simulator ยท Verifiable Evaluation ยท Scalable Online RL Training
MigoXLab / dingo
Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
FluxVLA / FluxVLA
An all-in-one VLA engineering platform for embodied AI โ from data to real-robot deployment.
TIGER-AI-Lab / VLM2Vec
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
baidu-baige / LoongForge
A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs. Supports Pi0.5, GR00T-N1.7, FastWAM, Cosmos3, DreamZero, DeepSeek-V4, GLM-5.3, Kimi-K3 and more.
RobotecAI / rai
RAI is a vendor agnostic agentic framework for Physical AI robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
NVIDIA / TensorRT-Edge-LLM
High-performance, light-weight C++ LLM and VLM Inference Software for Physical AI
remyxai / VQASynth
Compose multimodal datasets ๐น
zenstory-ai / video-recap-skills
Claude Code / Codex skills that turn a video into a Chinese narration recap (่ง้ข่งฃ่ฏด): scene detection, ASR, VLM, script, TTS, ffmpeg assembly, optional editable JianYing / CapCut draft export (ๅชๆ ่็จฟๅฏผๅบ). Local ffmpeg + one MiMo key, no GPU. | ็จ Claude Code skills ๆ่ง้ขๅๆไธญๆ่งฃ่ฏดๆ็๏ผๅฏ้ไธ้ฎๅฏผๅบๅฏ็ผ่พๅชๆ ่็จฟใ
showlab / Show-Harness
Just a VLM Agent Can Play Robots
SearchSavior / OpenArc
Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.
cheng-haha / GPT-Policy
GPT-Policy: In-Context Robot Learning with VLM Agents
zzzz7788990213-ops / EvoVLM
Evolving Multimodal Inference Programs: LLM-guided evolutionary search for accurate and efficient vision-language inference.
zqc3117 / Spotter
Official codebase for Spotter: Let the Embodied Model Lead, and the VLM Reflect for It
OmniJev / OneJev
๐๐ A multimodal System One decision model that gives calibrated answers to typed questions about screens, photos, video and text in one forward pass.
dimentary / llm-robotics-playground
A library of experiments using frontier LLMs/VLMs for robotics tasks, currently focusing on GPT-6 Astra.
trieuntu / VietAIDetector
VietAIDetector: A robust, zero-shot AI-generated text detection system for Vietnamese. Built on the VietBinoculars algorithm with PhoGPT-4B and featuring Vintern-1B VLM for scanned PDF OCR. Optimized for dual-GPU environments with a Gradio Web UI.
libnyx / LT2MD
AI-agent skill producing reusable Markdown from PDFs. It turns flowcharts, diagrams, and charts into text beside each caption instead of empty links. It checks an earlier conversion against the PDF and fixes misread or missing parts. Long PDFs run in small saved batches with an independent review pass, each paragraph tagged with its page.
drpwchen / lecture-to-notes
Lecture recordings โ structured grounded notes + a synced HTML viewer: video, timestamped transcript and curated summary on one page. Local GPU pipeline (Whisper ASR ยท slide extraction ยท OCR ยท VLM signals ยท capture-time alignment). Claude Code skill + plain CLI.
To3akaRin / VLMForge
Evaluate visual models on your own images, JSON Schema, and production constraints with LangGraph, Pareto analysis, and a no-key Replay demo.
benjiyaya / ComfyUI-H3-VisionPromptor
Feed an idea (plus optional reference images) and get Minimax H3 prompt โ with shots, camera vocabulary, dialogue tags, soundscape, and music fields โ generated entirely by a local VLM running on ComfyUI's native text-generation. No API keys, no cloud calls, no extra pip packages.
dk2472780158-ctrl / piper-dual-arm-smolvla-towel-folding
SmolVLA (SmolVLM2-500M) vision-language-action on real dual AgileX Piper arms โ continuous authority blending on a proven ACT safety base folds a real towel. One command runs the full 750-step skill (status=0). Full illustrated course included
LXnzh / VLM-grasp-sorting
Modular perception, grasp planning, tracking, and sorting pipeline for ML robotics
apple-aiml-research / ml-lensvlm
Official code for LensVLM: Selective Context Expansion for Compressed Visual Representation of Text.
r33drichards / laya-vision
Image inputs for Laya: calibrated, non-generative typed decisions over images + text (SmolVLM-256M backbone)
raullenchai / jetson-the-hard-way
Flash an old-firmware Jetson Orin Nano to JetPack 7.2 on NVMe with only a Mac + one USB stick โ no x86 host, no SD card. Firmware workarounds + local VLM/camera/PyTorch scripts + honest benchmarks.
Weschera / Qwen3.8-27B-oMLX-MTP-Mac
Qwen 3.8 27B at 48-65 tok/s on a Mac Studio M4 Max โ measured oMLX native-MTP recipe, same-day A/B vs mlx-vlm+MTP sidecar, raw benchmark JSON included
TianLin0509 / img2ppt-lite
Turn a slide mockup image into an editable PPTX: native text + semantic draggable sprites + inpainted background. Agent-as-VLM skill for Claude Code / Codex / any CLI agent.
Hearmeman24 / ComfyUI-MiniMaxRefPack
Reference management, portable configs and VLM prompt writing for ComfyUI's MiniMax H3 Reference-to-Video.