AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC
Run MiniMax H3 at native fp16 on pre-bf16 GPUs (V100/Volta) β exact-math fix for the black-frame overflows, ~11Γ faster than the fp32 fallback.
Mixed-precision acceleration patch for MiniMax H3 inference on NVIDIA V100
ComfyUI nodes for MiniMax H3: INT8 X2 VAE and asynchronous NVENC saving. 5-second 1088x1920 video with audio in about 60 s on an RTX 4070 12GB, about 30% faster than the FP16 X2 VAE.
Sol-Attn (arXiv 2607.24027) sparse acceleration for MiniMax H3 on V100 β single-node ComfyUI plugin (embedded FP16Safe + keep-or-drop kernel)