A framework for efficient model inference with omni-modality models
community maintained vllm/vllm-omni style inference engine for system1 models in Rust for extreme efficient performance
Measured SM121 compatibility recipe for MiniMax H3 FL2VA on one NVIDIA DGX Spark with vLLM-Omni and online FP8.
β
High-performance OpenAI and Anthropic compatible omni-modality model inference server for Apple Silicon.
β