HUANGCHIHHUNGLeo / claude-real-video
Let Claude (or any LLM) actually watch a video — scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.
HUANGCHIHHUNGLeo / claude-real-video
google-gemini / genai-processors
GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.
shcherbak-ai / contextgem
Extract structured data and insights from documents using LLMs and decision models
kyegomez / BitNet
Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
NVIDIA-AI-Blueprints / video-search-and-summarization
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
Core-Mate / OpenGUI
OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.
Tencent / WeMM-Embedding
WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.
facebookresearch / multimodal
TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.
bilibili / Index-Translate
A Multilingual, Multimodal Translation Model Family
pixeltable / pixeltable
The backend agents build with - Multimodal database, orchestration, and serving in one file
UbiquitousLearning / mllm
Fast Multimodal LLM on Mobile Devices
waybarrios / vllm-mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
DEEIX-AI / DEEIX-Chat
An enterprise AI workspace for model routing, multimodal chat, files, tools, billing, identity, and operations.
sgl-project / sglang-omni
SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.
emcf / thepipe
Get clean data from tricky documents, powered by vision-language models ⚡
NPC-Worldwide / npcpy
The python library for research and development in NLP, multimodal LLMs, Agents, ML, Knowledge Graphs, and more.
mattmireles / gemma-tuner-multimodal
Fine-tune Gemma 4 and 3n with audio, images and text on Apple Silicon, using PyTorch and Metal Performance Shaders.
zhaochen0110 / Awesome_Think_With_Images
Resources and paper list for "Thinking with Images for LVLMs". This repository accompanies our survey on how LVLMs can leverage visual information for complex reasoning, planning, and generation.
limecloud / lime
Full-stack AI agent for coding, files, terminals, tools, research, content, multimodal work, and multi-agent workflows.
Sun-Haoyuan23 / Awesome-RL-based-Reasoning-MLLMs
This repository provides valuable reference for researchers in the field of multimodality, please start your exploratory travel in RL-based Reasoning MLLMs!
llm-jp / awesome-japanese-llm
日本語LLMまとめ - Overview of Japanese LLMs
yuezhiai / jonex
All-in-One Multimodal Parsing Engine + Ontology-Powered, LLM Wiki-Driven AI-Ready Knowledge Engine
jordanrendric / claude-video-vision
Give Claude the ability to watch and understand videos — Claude Code plugin with frame extraction and multimodal audio analysis
xtreme1-io / xtreme1
Xtreme1 is an all-in-one data labeling and annotation platform for multimodal data training and supports 3D LiDAR point cloud, image, and LLM.
Capsize-Games / airunner
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
verl-project / verl-omni
Multimodal RL training framework for diffusion & omni models
Anionex / agent-vision-toolkit
为纯文本模型"看图“设计更好的视觉工具箱和技能，支持多图理解，图片问答，前端UI还原、GUI 自动化等，并可选无缝接入多个主流agent，直接识别粘贴图片｜ A vision toolkit and skill designed for text-only llms — image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode
EvolvingLMMs-Lab / LLaVA-OneVision-2
Fully Open Framework for Democratized Multimodal Training
Zen4-bit / Proxima
Multi-AI MCP Server - Connect ChatGPT, Claude, Gemini & Perplexity to your coding tools without any API
ALwrity / ALwrity
ALwrity - AI-first Digital Marketing Platform. AI Content Strategy and Planning, Multimodal content generation, Publishing, Analytics, AI SEO, Connect & Manage Social Accounts. Marketing OS - WIP
yashab-cyber / opendroid
Your Open Autonomous Android Agent — A production-ready, self-planning AI assistant powered by local/remote LLMs and accessibility-driven screen automation.
lhotse-speech / lhotse
Tools for handling multimodal data in machine learning projects.
ysr666 / dsh-vision-router
Eyes for text-only DeepSeek Harness agents: built-in free vision chain (no key) + pixel-level vision tools (Q&A, grounding, crop, pixel diff, colors, OCR, SVG trace, cutout, screenshots). One-command install, no Python, image turns work like ordinary tool-calling turns.
TheShadow29 / awesome-grounding
awesome grounding: A curated list of research papers in visual grounding
OpenMOSS / MOVA
A foundation model that generates synchronized video and audio in a single model
tong-io / tongflow
Modality-First GenAI Platform
983632847 / Awesome-Multimodal-Object-Tracking
A continuously updated project to track the latest progress in the field of multi-modal object tracking. This project focuses solely on single-object tracking.
dtsola / xiaoyaosearch
小遥搜索，听懂你的话、看懂你的图，用AI找到本地任何文件。让搜索像聊天一样简单。XiaoyaoSearch: Understands your words, reads your images, finds any local file with AI. Making search as easy as chatting.
yaotingwangofficial / Awesome-MCoT
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
Tencent-Hunyuan / UniRL
UniRL is a Framework for Unified Multimodal Model Reinforcement Learning
TinyLLaVA / TinyLLaVA_Factory
A Framework of Small-scale Large Multimodal Models
MinishLab / semhash
Fast Multimodal Semantic Deduplication & Filtering
livekit / agents-js
Build realtime multimodal AI agents with Node.js
Yuliang-Liu / MultimodalOCR
On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)
lightly-ai / lightly-studio
LightlyStudio - The Unified Data Platform for Multimodal ML
Valen-Team / Valen
Train a Jev-like multimodal model by yourself. System One Model, now with vision.
EvolvingLMMs-Lab / lmms-engine
A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.
whitecircle / halo
Halo is an open-source framework built by White Circle for training large language and multimodal models
Haervwe / open-webui-tools
Open‑WebUI Tools is a modular toolkit designed to extend and enrich your Open WebUI instance, turning it into a powerful AI workstation. With a suite of over 15 specialized tools, function pipelines, and filters, this project supports academic research, agentic autonomy, multimodal creativity, workflows, and more
suzuran0y / CCTV-Smartphone-AI-Monitoring
本地监控 + AI 视觉 — LAN-based smartphone-powered AI monitoring framework with structured event output for data acquisition and analysis.
OpenMOSS / MOSS-VL
An open-weight 11B model series for long-form and real-time video understanding
Flaminis / Dalaran
Dalaran — Apache-2.0, robotics-first visualization and data infrastructure for multimodal time-series. ROS 2 native, reads existing .rrd recordings. A hard fork of Rerun.
zli12321 / Vision-Language-Models-Overview
A most Frontend Collection and survey of vision-language model papers, and models GitHub repository. Continuous updates.
magicrew / doc7
Turn documents into AI-ready Markdown with visual understanding
aimclub / FEDOT
Automated modeling and machine learning framework FEDOT
TIGER-AI-Lab / VLM2Vec
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
TeleAI-UAGI / Awesome-Agent-Memory
Curated systems, benchmarks, and papers etc. on memory for LLMs/MLLMs --- long-term context, retrieval, and reasoning.
henrydaum / second-brain
Second Brain is an agentic framework that acts as an operating system, using local file intelligence, workflow automation, and LLMs to complete tasks and communicate over multiple modalities and messaging platforms.
redai-studio / Relax
An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale
DenisovAV / flutter_edge_ai
On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.