The NVIDIA deep learning SDK offers powerful tools and libraries for the development of deep learning frameworks such as Caffe, CNTK, TensorFlow, Theano, and Torch. The NVIDIA deep learning SDK provides powerful tools and libraries for designing and deploying GPU-accelerated deep learning applications. It includes libraries for deep learning primitives, inference, video analytics, linear algebra, sparse matrices, and multi-GPU communications. The current implementation supports the following SDKs:
- Deep learning primitives: (https://developer.nvidia.com/cudnn) High-performance building blocks for deep neural network applications including convolutions, activation functions, and tensor transformations.
- Deep learning inference engine: (https://developer.nvidia.com/tensorrt) High-performance deep learning inference runtime for production deployment.
- Deep learning for video analytics: (https://developer.nvidia.com/deepstream-sdk) High-level C++ API and runtime for GPU-accelerated transcoding and deep learning inference.
- Linear algebra: (https://developer.nvidia.com/cublas) GPU-accelerated BLAS functionality that delivers 6x to 17x faster performance than CPU-only BLAS libraries. The XLA (https://www.tensorflow.org/performance/xla/) is a domain-specific compiler for linear algebra that optimizes TensorFlow computations. Although this experimental (that is, under active development), however, the possible results are improvements in speed, memory usage, and portability on server and mobile platforms.
- Sparse matrix operations: (https://developer.nvidia.com/cusparse) GPU-accelerated linear algebra subroutines for sparse matrices that deliver up to 8x faster performance than CPU BLAS (MKL), ideal for applications such as natural language processing.
- Multi-GPU communication: (https://github.com/NVIDIA/nccl) Collective communication routines, such as all-gather, reduce, and broadcast that accelerate multi-GPU deep learning training on up to eight GPUs.
However, the deep learning SDK requires CUDA toolkit (https://developer.nvidia.com/cuda-toolkit), which offers a comprehensive development environment for building new GPU-accelerated deep learning algorithms, and dramatically increasing the performance of existing applications.
To use TensorFlow with NVIDIA GPUs, the first step is to install the CUDA toolkit.
Once the CUDA toolkit is installed, you must download the cuDNN v5.1 library from
https://developer.nvidia.com/cudnn for Linux.
A more detailed installation including TensorFlow and Bazel for GPU computation using cuDNN, please refer to this URL http://www.nvidia.com/object/gpu-accelerated-applications-tensorflow-installation.html.
The cuDNN is a library that helps accelerate deep learning frameworks, such as TensorFlow or Theano. Here's a brief explanation from the NVIDIA website.
The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for DNNs. The cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. The cuDNN is part of the NVIDIA deep learning SDK.
Before installing you'll need to register for NVIDIA's Accelerated Computing Developer Program. Once registered, login and download CudNN 5.1 to your local computer.
During the writing of this book, the version of cuDNN was 5.1 and released on Jan 20, 2017, for CUDA 8.0. For more details at https://developer.nvidia.com/rdp/cudnn-download. You should see options in the downlaod page as shown in the following figure:
As shown in the preceding figure, you will have to select your platform/OS type. The following installation is for Linux. Now, once downloaded, uncompress the files and copy them into the CUDA toolkit directory (assumed here to be in /usr/local/cuda/):
$ sudo tar -xvf cudnn-8.0-linux-x64-v5.1-rc.tgz -C /usr/local