FareedKhan-dev / kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
FareedKhan-dev / kimi-k3-in-c
intel / auto-round
PTQ-Level Efficiency, QAT-Level Accuracy|训练后量化的效率，量化感知训练的精度
truespar / paddock
Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio
tsaipifong / whirl-llm
WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.
intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
microsoft / Tutel
Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4
GGZ14 / vllm-mxfp4
The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper