bilibili / Index-Translate
A Multilingual, Multimodal Translation Model Family
bilibili / Index-Translate
pixeltable / pixeltable
The backend agents build with - Multimodal database, orchestration, and serving in one file
waybarrios / vllm-mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
autonomousvision / transfuser
[PAMI'23] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving; [CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
sgl-project / sglang-omni
SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.
emcf / thepipe
Get clean data from tricky documents, powered by vision-language models ⚡
NPC-Worldwide / npcpy
The python library for research and development in NLP, multimodal LLMs, Agents, ML, Knowledge Graphs, and more.
mattmireles / gemma-tuner-multimodal
Fine-tune Gemma 4 and 3n with audio, images and text on Apple Silicon, using PyTorch and Metal Performance Shaders.
yuezhiai / jonex
All-in-One Multimodal Parsing Engine + Ontology-Powered, LLM Wiki-Driven AI-Ready Knowledge Engine
Capsize-Games / airunner
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
verl-project / verl-omni
Multimodal RL training framework for diffusion & omni models
modal-labs / modal-examples
Examples of programs built using Modal
Anionex / agent-vision-toolkit
为纯文本模型"看图“设计更好的视觉工具箱和技能，支持多图理解，图片问答，前端UI还原、GUI 自动化等，并可选无缝接入多个主流agent，直接识别粘贴图片｜ A vision toolkit and skill designed for text-only llms — image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode
EvolvingLMMs-Lab / LLaVA-OneVision-2
Fully Open Framework for Democratized Multimodal Training
ALwrity / ALwrity
ALwrity - AI-first Digital Marketing Platform. AI Content Strategy and Planning, Multimodal content generation, Publishing, Analytics, AI SEO, Connect & Manage Social Accounts. Marketing OS - WIP
lhotse-speech / lhotse
Tools for handling multimodal data in machine learning projects.
OpenMOSS / MOVA
A foundation model that generates synchronized video and audio in a single model
dtsola / xiaoyaosearch
小遥搜索，听懂你的话、看懂你的图，用AI找到本地任何文件。让搜索像聊天一样简单。XiaoyaoSearch: Understands your words, reads your images, finds any local file with AI. Making search as easy as chatting.
Tencent-Hunyuan / UniRL
UniRL is a Framework for Unified Multimodal Model Reinforcement Learning
TinyLLaVA / TinyLLaVA_Factory
A Framework of Small-scale Large Multimodal Models
MinishLab / semhash
Fast Multimodal Semantic Deduplication & Filtering
Yuliang-Liu / MultimodalOCR
On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)
lightly-ai / lightly-studio
LightlyStudio - The Unified Data Platform for Multimodal ML
InternLM / Intern-Decision
Fast multi-modal decision model
EvolvingLMMs-Lab / lmms-engine
A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.
whitecircle / halo
Halo is an open-source framework built by White Circle for training large language and multimodal models
Haervwe / open-webui-tools
Open‑WebUI Tools is a modular toolkit designed to extend and enrich your Open WebUI instance, turning it into a powerful AI workstation. With a suite of over 15 specialized tools, function pipelines, and filters, this project supports academic research, agentic autonomy, multimodal creativity, workflows, and more
suzuran0y / CCTV-Smartphone-AI-Monitoring
本地监控 + AI 视觉 — LAN-based smartphone-powered AI monitoring framework with structured event output for data acquisition and analysis.
aimclub / FEDOT
Automated modeling and machine learning framework FEDOT
TIGER-AI-Lab / VLM2Vec
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]
TeleAI-UAGI / Awesome-Agent-Memory
Curated systems, benchmarks, and papers etc. on memory for LLMs/MLLMs --- long-term context, retrieval, and reasoning.
decodingai-magazine / building-a-coding-agent-from-scratch-course
Learn harness engineering by building Claude Code from scratch. Free, open-source course: 8 articles, 6 videos, 1 codebase.
henrydaum / second-brain
Second Brain is an agentic framework that acts as an operating system, using local file intelligence, workflow automation, and LLMs to complete tasks and communicate over multiple modalities and messaging platforms.
redai-studio / Relax
An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale
ggozad / haiku.rag
Agentic RAG for local and self-hosted document search: hybrid retrieval, reranking and multimodal RAG on embedded LanceDB, with Docling parsing and an MCP server
ZebangCheng / Emotion-LLaMA
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
SWE-agent / SWE-ReX
Sandboxed code execution for AI agents, locally or on the cloud. Massively parallel, easy to extend. Powering SWE-agent and more.
RobotecAI / rai
RAI is a vendor agnostic agentic framework for Physical AI robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
remyxai / VQASynth
Compose multimodal datasets 🎹
AIFrontierLab / TorchUMM
A unified multimodal model toolkit (NeurIPS'26)
rockmatrix-ai / EdgeRock
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
modal-labs / modal-client
SDK libraries for Modal
opengeos / GeoAgent
A multimodal AI agent for geospatial data analysis and interactive visualization
oxbshw / watch-skill
Give AI agents eyes, ears, and verifiable results. Watch Skill turns video, audio and screen activity into searchable, timestamped evidence and proves work with deterministic contracts, not model opinion. DeepWatch is the agent workspace built on DeepSeek Harness. Python + npm, MCP, CLI, REST, Web.
v-modal / vmodal_sdk_robotics
Robotics vision SDK
Code-Amadeus / Amadeus
Real-time multimodal desktop agent evolving toward a persistent AI OS interface (0.17 α).
PacesaLab / BindCraft2
Easy to use suite for designing a wide range of protein binder modalities.
OliverDOU776 / Few-step-probabilistic-glucose-forecasting-from-continuous-glucose-monitoring-and-meal-images
Official research code for GlucoFlow: few-step multimodal probabilistic glucose forecasting.
TaoLiveAIGC / TLive-Omni
TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming
PipeNetwork / kimi-k3-mlx
MLX port of moonshotai/Kimi-K3 (2.78T multimodal MoE): streaming converter, REAP expert pruning, and per-language expert-overlap analysis
zzzz7788990213-ops / EvoVLM
Evolving Multimodal Inference Programs: LLM-guided evolutionary search for accurate and efficient vision-language inference.
six-nut / PocketMen-with-you
Create high-fidelity Codex companions from 2+ photos with local open-weight neural editing and no OpenAI API key.
Hebbian-Robotics / hflow
SDK for robotics teams to verify the quality of their data used for AI model training.
openaiotlab / CUHK-X
[MobiSys 2026] A large-scale, multimodal dataset and benchmark for Human Action Recognition, Understanding and Reasoning
znxzsy / VeriInk
Verifiable handwriting synthesis for failure-driven multimodal training and regression.
rollroyces / py-idp
AI-enabled Intelligent Document Processing for Python: 12+ LLM backends, Pydantic schemas, self-hosted OCR (Nanonets-OCR2-3B), auto-schema discovery, HITL review, prod FastAPI server. AGPL-3.0 + commercial.
duckyshell / ComfyUI-MiniMaxH3-Prompt-Writer
Multimodal MiniMax H3 prompt writer for ComfyUI with local and API providers.
LynnReal-AI / LynnReal-Omni
LynnReal-Omni brings text-to-video, image-to-video, human- and hand-pose guided generation, structural control, omni-reference generation, style transfer, video editing, degraded-video restoration and streaming long-video generation into a single framework, all at four-step fast generation.
B0yko / tern
Local AI search for podcast and video archives on Apple Silicon: Whisper speech, Apple Vision OCR and SigLIP-2 visual search fused into one ranked list, with a trim editor and FCPXML export. FastAPI, vanilla JS, Tauri. Source-visible, all rights reserved.
MoonshotAI / PerceptionBench
PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models