Trainable fast and memory-efficient sparse attention
We propose Prism, a dynamic sparse attention framework for natively training joint video-audio generation models at 2K.
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention
Veda Sparse Attention for ComfyUI
NVIDIA Sol-Attn for ComfyUI / Triton kernel on SM89 - SM121, with zero-copy MiniMax H3 nodes: memory-efficient attention, scheduled tau with graph preview, and feed-forward chunking. Measured 1.14–1.44× vs SageAttention and −37% MLP peak VRAM on H3
Adaptive Block Sparse Attention for MiniMax-H3
GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode
Sparse acceleration and LoRA fine-tuning of MiniMax-H3 on resource-constrained machines.
Piecewise-Taylor Attention
This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.
Pyramis-L3D: hierarchical sparse attention research prototype (code only; weights on Hugging Face)