NVIDIA / nccl

# NCCL
Optimized primitives for inter-GPU communication.
## Introduction
NCCL (pronounced "Nickel") is a stand-alone library of standard communication routines for GPUs, implementing all-reduce, all-gather, reduce, broadcast, reduce-scatter, as well as any send/receive based communication pattern. It has been optimized to achieve high bandwidth on platforms using PCIe, NVLink, NVswitch, as well as networking using InfiniBand Verbs or TCP/IP sockets. NCCL supports an arbitrary number of GPUs installed in a single node or across multiple nodes, and can be used in either single- or multi-process (e.g., MPI) applications.
For more information on NCCL usage, please refer to the NCCL documentation.
## Build
Note: the official and tested builds of NCCL can be downloaded from: https://developer.nvidia.com/nccl. You can skip the following build steps if you choose to use the official builds.
To build the library :
 [code] 
If CUDA is not installed in the default /usr/local/cuda path, you can define the CUDA path with :
 [code] 
NCCL will be compiled and installed in `build/` unless `BUILDDIR` is set.
By default, NCCL is compiled for all supported architectures. To accelerate the compilation and reduce the binary size, consider redefining `NVCC_GENCODE` (defined in `makefiles/common.mk`) to only include the architecture of the target platform :
 [code] 
## Install
To install NCCL on the system, create a package then install it as root.
Debian/Ubuntu :
 [code] 
RedHat/CentOS :
 [code] 
OS-agnostic tarball :
 [code] 
Python wheel :
 [code] 
## Tests
Tests for NCCL are maintained separately at https://github.com/nvidia/nccl-tests.
 [code] 
## Copyright
All source code and accompanying documentation is copyright (c) 2015-2020, NVIDIA CORPORATION. All rights reserved.