NVIDIA / Model-Optimizer

# NVIDIA Model Optimizer
Documentation |
Roadmap |
Announcement Blogs
______________________________________________________________________
**NVIDIA Model Optimizer** (referred to as **Model Optimizer**, or **ModelOpt**) is a library comprising state-of-the-art model optimization techniques including quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding and sparsity to accelerate models.
**[Input]** Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model.
**[Optimize]** Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint.
Model Optimizer is also integrated with NVIDIA Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques.
**[Export for deployment]** Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM. The unified Hugging Face export API now supports both transformers and diffusers models.
## Latest News
- [2026/10/15] NVIDIA Nemotron 3.5 Super VL 120B-A12B NVFP4 checkpoint: Quantized with Model Optimizer using post-training quantization (PTQ) and quantization-aware distillation (QAD). For details, see the Quantization section (Section 4) of the technical report.
- [2026/10/08] BLOG: AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization: Per-layer KV format search, GEMM composition, and a Qwen3-8B accuracy case study.
- [2026/09/16] BLOG: Scale Learning: Training NVFP4 Block Scales to Recover Accuracy
- [2026/09/16] **End-to-end W4A4 NVFP4 + QAD tutorial for Qwen3.6-35B-A3B**: NVFP4 W4A4 PTQ plus quantization-aware distillation, reaching up to 1.30x vLLM throughput over BF16 and 3.1x smaller checkpoints while recovering the accuracy W4A4 costs.
- [2026/09/09] BLOG: Improving NVFP4 Accuracy with Local-Hessian Weight Scales
- [2026/08/24] BLOG: AutoQuantize: A Fast Automatic Mixed-Precision Assignment
- [2026/08/17] BLOG: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer: Learn how quantization-aware distillation recovers accuracy from aggressive NVFP4 quantization while reducing model size and increasing throughput.
- [2026/06/26] BLOG: Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer: How we quantized Nemotron 3 Ultra (550B) to NVFP4 with Model Optimizer — up to 5.9× higher decode-heavy inference throughput than GLM-5.1 754B FP4 while matching BF16 accuracy. NVFP4 Checkpoint on Hugging Face.
- [2026/05/27] **End-to-end Optimization tutorial for Nemotron-3-Nano-30B-A3B**: Pruning + two-phase distillation + FP8 quantization achieving 2.6× vLLM throughput and 2.6× memory reduction.
- [2026/05/13] **Puzzletron**: A new algorithm for heterogeneous pruning & NAS of LLM and VLM models.
- [2026/04/15]