NVIDIA / cutlass

CUDA C++ templates and Python DSLs for building high-performance GEMM and other linear algebra kernels on NVIDIA GPUs.

NVIDIA / nccl

NVIDIA's C++/CUDA library of optimized collective communication primitives (all-reduce, all-gather, broadcast, etc.) for multi-GPU and multi-node systems.