Build and run agents you can see, understand and trust.
See, hear, and interact on your local devices.
"RAG-Anything: All-in-One RAG Framework"
:sparkles::sparkles:Latest Advances on Multimodal Large Language Models
An open source implementation of CLIP.
Open-source framework for conversational voice AI agents
ModelScope: bring the notion of Model-as-a-Service to life.
Data processing for and with foundation models! π π π½ β‘οΈ β‘οΈπΈ πΉ π·
AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. Includes AI personas, AGI functions, world-class Beam multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.
Open Source Routing Engine for OpenStreetMap
A comprehensive list of papers using large language/multi-modal models for Robotics/RL, including papers, codes, and related websites
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
Superfast AI decision making and intelligent processing of multi-modal data.
A C#/.NET library to run LLM (π¦LLaMA/LLaVA) on your local device efficiently.
Open, Multi-modal Catalog for Data & AI
Collection of AWESOME vision-language models for vision tasks
An open source multi-modal trip planner
Monkey (LMM): Image Resolution and Text Label Are Important Things for Large Multi-modal Models (CVPR 2024 Highlight)
A robust, all-in-one GPT interface for Discord. ChatGPT-style conversations, image generation, AI-moderation, custom indexes/knowledgebase, youtube summarizer, and more!
Local-first Video Knowledge Base. Index your video library with multi-modal analysis (YOLO, DeepFace, Whisper), search semantically via natural language, Docker-ready.
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.
[PAMI'23] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving; [CVPR'21] Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
SALMONN family: A suite of advanced multi-modal LLMs
A continuously updated project to track the latest progress in the field of multi-modal object tracking. This project focuses solely on single-object tracking.
Fast multi-modal decision model
π Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
β
ποΈ Real-Time Multi-Modal Speech Telemetry & Acoustic DSP Engine (FastAPI, Python 3.11, Autocorrelation Pitch, FFT Spectral Energy & Lexical Pacing Analytics).
The Official Implementation of MMDiff: Multi-modal Remote Sensing Image Generation via Cross-Modality Spatial Feature Transfer
Spatial Real-time Traffic Processing Optimizer for Intelligent Distributed Multi-Modal Transportation Network Planner.