Agent S: the state-of-the-art open source computer use agents
Curated systems, benchmarks, and papers etc. on memory for LLMs/MLLMs --- long-term context, retrieval, and reasoning.
β
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
Turn local language models into Jev-style structured decision models. Get results from text and images with prefill aloneβno token-by-token decoding required.
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
This repository provides valuable reference for researchers in the field of multimodality, please start your exploratory travel in RL-based Reasoning MLLMs!
Fully Open Framework for Democratized Multimodal Training
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. ACM Computing Surveys, 2026.
This is a repository for listing papers on scene graph generation and application.
Cell type annotation for single-cell RNA-seq using multi-LLM consensus
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
Awesome Multimodal Modeling [Covers MLLM, UMM, and NMM]
A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..
β
π¬ OraRL β Annotations as Rollouts for efficient, scalable reinforcement learning of unified video MLLMs.
Repo of VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.
Network-faithful simulation of LLM serving and training deployments
Self-hosted AI platform on Apple Silicon β production-proven, Thai-optimized, $3.50/day
Official implementation of "[NeurIPS'2026] Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering"
β
Official implementation of "DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception"
Fast Multimodal LLM on Mobile Devices