alirezadir / AIMLInterviews
A Comprehensive Guide for Generative AI, Agentic AI, Multimodal AI, Physical AI Interviews. Proven Formula for multiple FAANG offers and more
alirezadir / AIMLInterviews
uclanlp / OpenVLHarness
Official Code for OpenVLHarness: Real-World Visual Agentic Tool‑Use with Multimodal Memory
huggingface / transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Mintplex-Labs / anything-llm
Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience
bojieli / ai-agent-book
《深入理解 AI Agent：设计原理与工程实践》（李博杰 著）开源主仓库：全书正文、编译版 PDF 与按章配套代码
bytedance / UI-TARS-desktop
The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
sgl-project / sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
deepset-ai / haystack
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.
microsoft / unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
screenpipe / screenpipe
YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
QwenLM / Qwen3-VL
Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
NVIDIA-NeMo / Speech
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
BradyFU / Awesome-Multimodal-Large-Language-Models
:sparkles::sparkles:Latest Advances on Multimodal Large Language Models
pipecat-ai / pipecat
Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.
modelscope / ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
multimodal-art-projection / YuE
YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.
datawhalechina / all-in-rag
🔍大模型应用开发实战一：RAG 技术全栈指南，在线阅读地址：https://datawhalechina.github.io/all-in-rag/
lancedb / lancedb
Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage Less.
rerun-io / rerun
Visualize, query, and stream to train on multimodal robotics data.
CVHub520 / X-AnyLabeling
X-AnyLabeling: A lightweight, efficient, and unified cross-platform desktop application for annotating text, image, video, point cloud and multimodal data, combining versatile built-in tools with state-of-the-art AI models and flexible multi-format export.
RunanywhereAI / runanywhere-sdks
Production ready toolkit to run AI locally
StarTrail-org / PixelRAG
https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/
OthersideAI / self-operating-computer
A framework to enable a multimodal model to operate a computer.
yzhao062 / pyod
A Python library for anomaly detection across tabular, time series, graph, text, image, and audio data. 60+ detectors, benchmark-backed ADEngine orchestration, and an agentic workflow for AI agents.
gorse-io / gorse
AI powered open source recommender system engine supports classical/LLM rankers and multimodal content via embedding
apache / seatunnel
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
xorbitsai / inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
bentoml / BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
AI4Finance-Foundation / FinRobot
FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models
Blaizzy / mlx-audio
A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.
vllm-project / vllm-omni
A framework for efficient model inference with omni-modality models
lance-format / lance
Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
genkit-ai / genkit
Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google
Eventual-Inc / Daft
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
OpenBMB / UltraRAG
A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines
facebookresearch / mmf
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
ARahim3 / DigUp
Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.
InternLM / xtuner
A Next-Generation Training Engine Built for Ultra-Large MoE Models
kyegomez / tree-of-thoughts
Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%
crmne / ruby_llm
The Ruby AI framework.
EvolvingLMMs-Lab / lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
liustack / modlens
The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, get structured JSON evidence (OCR, layout, semantics). | 全网最强 DeepSeek Harness 外挂视觉插件，为 DeepSeek、GLM 等纯文本模型外挂视觉能力，粘贴图片即得结构化 JSON 证据（OCR、版面、语义）。
microsoft / PhiCookBook
This is a Phi Family of SLMs book for getting started with Phi Models. Phi a family of open sourced AI models developed by Microsoft. Phi models are the most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up across a variety of language, reasoning, coding, and math benchmarks
morphik-org / morphik-core
Open-source multimodal retrieval engine (Morphik Core)
embeddings-benchmark / mteb
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
nicedreamzapp / claude-code-local
Run Claude Code 100% on-device with local AI on Apple Silicon. MLX-native Anthropic-API server. 6 fighters incl. Muse-Glimmer 30B (now multimodal — reads images, abliterated), Gemma 4 31B, Qwen 3.5 122B (65 tok/s), DeepSeek V4 Flash (1M ctx). Private, offline, airgap-ready. Built for NDA / legal / healthcare workflows.
vortex-data / vortex
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.
lakesoul-io / LakeSoul
LakeSoul is an end-to-end, realtime cloud-native Lakehouse framework for fast data ingestion, concurrent updates, incremental analytics, multimodal data processing and vector search — powering next-generation BI and AI workloads.
xlang-ai / OSWorld
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
TanStack / ai
🤖 Type-safe, provider-agnostic TypeScript AI SDK for streaming chat, tool calling, agents, and multimodal apps across OpenAI, Anthropic, Gemini, React, Vue, Svelte, and Solid.
QwenLM / Qwen-MM-Plugins
Make any agent harness multimodal-native.
datachain-ai / datachain
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
autodistill / autodistill
Images to inference with no labeling (use foundation models to train supervised models).
roboflow / maestro
streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL
om-ai-lab / OmAgent
[EMNLP-2024] Build multimodal language agents for fast prototype and production
genieincodebottle / generative-ai
Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.
AI-Efficiency / Awesome-Model-Quantization
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
elevenlabs / ui
ElevenLabs UI is a component library and custom registry built on top of shadcn/ui to help you build multimodal agents faster.
zai-org / GLM-V
GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
NVIDIA-NeMo / DataDesigner
🎨 NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.