ivandilelio / kybes
Know Your Backend is a Plugin that help your AI to generate a structured Model/View/Controller application, witouth surprise, code duplication and different modality in a different project
ivandilelio / kybes
uclanlp / OpenVLHarness
Official Code for OpenVLHarness: Real-World Visual Agentic ToolโUse with Multimodal Memory
huggingface / transformers
๐ค Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
bojieli / ai-agent-book
ใๆทฑๅ ฅ็่งฃ AI Agent๏ผ่ฎพ่ฎกๅ็ไธๅทฅ็จๅฎ่ทตใ๏ผๆๅๆฐ ่๏ผๅผๆบไธปไปๅบ๏ผๅ จไนฆๆญฃๆใ็ผ่ฏ็ PDF ไธๆ็ซ ้ ๅฅไปฃ็
sgl-project / sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
agentscope-ai / agentscope
Build and run agents you can see, understand and trust.
OpenBMB / MiniCPM-V
See, hear, and interact on your local devices.
deepset-ai / haystack
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.
HKUDS / RAG-Anything
"RAG-Anything: All-in-One RAG Framework"
microsoft / unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
NVIDIA-NeMo / Speech
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
pipecat-ai / pipecat
Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.
modelscope / ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
mlfoundations / open_clip
An open source implementation of CLIP.
multimodal-art-projection / YuE
YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.
datawhalechina / all-in-rag
๐ๅคงๆจกๅๅบ็จๅผๅๅฎๆไธ๏ผRAG ๆๆฏๅ จๆ ๆๅ๏ผๅจ็บฟ้ ่ฏปๅฐๅ๏ผhttps://datawhalechina.github.io/all-in-rag/
TEN-framework / ten-framework
Open-source framework for conversational voice AI agents
CVHub520 / X-AnyLabeling
X-AnyLabeling: A lightweight, efficient, and unified cross-platform desktop application for annotating text, image, video, point cloud and multimodal data, combining versatile built-in tools with state-of-the-art AI models and flexible multi-format export.
StarTrail-org / PixelRAG
https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
OthersideAI / self-operating-computer
A framework to enable a multimodal model to operate a computer.
yzhao062 / pyod
A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.
xorbitsai / inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop โ all through one unified, production-ready inference API.
modelscope / modelscope
ModelScope: bring the notion of Model-as-a-Service to life.
bentoml / BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
AI4Finance-Foundation / FinRobot
FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models
Blaizzy / mlx-audio
A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.
Emily2040 / seedance-2.0
Comprehensive production pipeline for quad-modal AI filmmaking with Seedance 2.0
vllm-project / vllm-omni
A framework for efficient model inference with omni-modality models
datajuicer / data-juicer
Data processing for and with foundation models! ๐ ๐ ๐ฝ โก๏ธ โก๏ธ๐ธ ๐น ๐ท
OpenBMB / UltraRAG
A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines
facebookresearch / mmf
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
InternLM / xtuner
A Next-Generation Training Engine Built for Ultra-Large MoE Models
ParisNeo / lollms-webui
Lord of Large Language and Multi modal Systems Web User Interface
kyegomez / tree-of-thoughts
Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%
EvolvingLMMs-Lab / lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
open-compass / VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
eclipse-sumo / sumo
Eclipse SUMO is an open source, highly portable, microscopic and continuous traffic simulation package designed to handle large networks. It allows for intermodal simulation including pedestrians and comes with a large set of tools for scenario creation.
aurelio-labs / semantic-router
Superfast AI decision making and intelligent processing of multi-modal data.
morphik-org / morphik-core
Open-source multimodal retrieval engine (Morphik Core)
embeddings-benchmark / mteb
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
nicedreamzapp / claude-code-local
Run Claude Code 100% on-device with local AI on Apple Silicon. MLX-native Anthropic-API server. 6 fighters incl. Muse-Glimmer 30B (now multimodal โ reads images, abliterated), Gemma 4 31B, Qwen 3.5 122B (65 tok/s), DeepSeek V4 Flash (1M ctx). Private, offline, airgap-ready. Built for NDA / legal / healthcare workflows.
xlang-ai / OSWorld
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
QwenLM / Qwen-MM-Plugins
Make any agent harness multimodal-native.
datachain-ai / datachain
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
autodistill / autodistill
Images to inference with no labeling (use foundation models to train supervised models).
roboflow / maestro
streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL
om-ai-lab / OmAgent
[EMNLP-2024] Build multimodal language agents for fast prototype and production
zai-org / GLM-V
GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
NVIDIA-NeMo / DataDesigner
๐จ NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.
ByteDance-Seed / VeOmni
VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
HUANGCHIHHUNGLeo / claude-real-video
Let Claude (or any LLM) actually watch a video โ scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.
google-gemini / genai-processors
GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.
fabiocaccamo / django-admin-interface
:superhero: :zap: django's default admin interface with superpowers - customizable themes, popup windows replaced by modals and many other features.
shcherbak-ai / contextgem
Extract structured data and insights from documents using LLMs and decision models
Yuliang-Liu / Monkey
Monkey (LMM): Image Resolution and Text Label Are Important Things for Large Multi-modal Models (CVPR 2024 Highlight)
kyegomez / BitNet
Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
NVIDIA-AI-Blueprints / video-search-and-summarization
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
Kav-K / GPTDiscord
A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!
Tencent / WeMM-Embedding
WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.
facebookresearch / multimodal
TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.