Getting Started

Dellve CuDNN is one of the tools that are used as an extension to the main project Dellve Deep on GitHub. This tool utilizes NVIDIA cuDNN operations as benchmark and stress test tools.

You can install Dellve CuDNN from our GitHub repo:

pip install git+https://github.com/dellve/dellve_cudnn.git

Install Requirements

Dellve CuDNN requires CUDA and CuDNN before setup.

Important

These environment variables must be set before installation:

CUDA_PATH: Path to CUDA library

CUDNN_PATH: Path to CUDNN library

CUDA Setup

CuDNN Setup

Dellve CuDNN Framework

This framework uses a cpp to python framework PyBind11 to create functions in cpp and allows them to be controlled by Python side. The CPP code uses an open source Python wrapper: PyCuDNN to make calls to CuDNN simpler.

class DELLve::BenchmarkController

Controller class that speficies the actual runtime execution of each tool in Dellve CuDNN. Provides functions that allows profiling of current status of the exeuction.

Assumes that BenchmarkDriver has been ran with the function of current operation as desired.

Subclassed by DELLve::BenchmarkDriver< A >

Public Functions

void startBenchmark(int deviceId, int numRuns)

Starts a benchmark tool.

Sets device to desired GPU. Warms up the test by running a singular operation and synchronizing. Then, runs the operation the specified number of times. Through each run, calculates time it took to run the current method and sets the progress of the current run by the number of loops ran.

This is done in a different thread so that external calls can be made to this class to profile the current status.

Parameters
  • deviceId: - Device ID to run test on
  • numRuns: - Number of repeats

void startStressTool(int deviceId, int seconds)

Starts a stress test tool.

Sets device to desired GPU. Starts the clock and runs the benchmark until desired elapsed time has been reached. Through each run, sets the progress of the test by the number of seconds elapsed.

This is also done in a separate thread so that external calls can be made to this class to profile the current status.

Parameters
  • deviceId: - Device ID to run test on
  • seconds: - Number of seconds to run the test

float getProgress() const

Returns the progress set by the tests.

Return
float - Progress of the current tool.

int getAvgTimeMicro() const

Returns the average time taken to run the current problem set in the benchmarks.

Should not be called by Stress Tools.

Return
int - Average time in microsends taken for each benchmark.

Dellve CuDNN Operations

Following are all of the operations supported by Dellve CuDNN.

template <typename T>
DELLve::Benchmark DELLve::Activation::forward(int w, int h, int c, int n)

CuDNN Activation Forward

Build 4D tensors using NCHW provided. Fill the input with random data using the cuRAND library. Then, set up the function that will run the operation with forward propagation.

Parameters
  • w: - Width of each feature map
  • h: - Height of each feature map
  • c: - Number of feature maps per image
  • n: - Number of feature maps

template <typename T>
DELLve::Benchmark DELLve::Activation::backward(int w, int h, int c, int n)

CuDNN Activation Backward

Build 4D tensors using NCHW provided. Fill the input with random data using the cuRAND library. Then, set up the function that will run the operation with backward propagation.

See
See forward for parameter details

template <typename T>
DELLve::Benchmark DELLve::Softmax::forward(int w, int h, int c, int n, std::string alg)

CuDNN Softmax Forward

Build 4D tensors using NCHW provided. Fill the input with random data using the cuRAND library. Then, set up the function that will run the operation with forward propagation with algorithm specified.

Parameters
  • w: - Width of each feature map
  • h: - Height of each feature map
  • c: - Number of feature maps per image
  • n: - Number of feature maps
  • alg: - Algorithm to run. Can be fast, accurate, or log

template <typename T>
DELLve::Benchmark DELLve::Softmax::backward(int w, int h, int c, int n, std::string alg)

CuDNN Softmax Backward

Build 4D tensors using NCHW provided. Fill the input with random data using the cuRAND library. Then, set up the function that will run the operation with backward propagation with algorithm specified.

See
See forward for parameter details

template <typename T>
DELLve::Benchmark DELLve::Convolution::forward(int w, int h, int c, int n, int k, int r, int s, int padW, int padH, int strideV, int strideH)

CuDNN Convolution Forward

Build 4D tensors using NCHW and KCRS provided for input and filter respectively. Then, create the output tensor by calculating the forward output dimensions of convolution. Finally, set up the workspace required and return the function that will run the operation with forward propagation.

Parameters
  • w: - Width of input image
  • h: - Height of input image
  • c: - Number of channels per input image
  • n: - Number of input images
  • k: - Number of filters
  • r: - Height of filter
  • s: - Width of filter
  • padW: - Width of zero padding
  • padH: - Height of zero padding
  • strideV: - Filter vertical stride
  • strideH: - Filter horizontal stride

template <typename T>
DELLve::Benchmark DELLve::Convolution::backwardData(int w, int h, int c, int n, int k, int r, int s, int padW, int padH, int strideW, int strideH)

CuDNN Convolution Backward Data

Build 4D tensors using NCHW and KCRS provided for input and filter respectively. Then, create the output tensor by calculating the forward output dimensions of convolution. Finally, set up the workspace required and return the function that will run the operation with backward propagation respective to data.

See
See forward for parameter details

template <typename T>
DELLve::Benchmark DELLve::Convolution::backwardFilter(int w, int h, int c, int n, int k, int r, int s, int padW, int padH, int strideW, int strideH)

CuDNN Convolution Backward Filter

Build 4D tensors using NCHW and KCRS provided for input and filter respectively. Then, create the output tensor by calculating the forward output dimensions of convolution. Finally, set up the workspace required and return the function that will run the operation with backward propagation respective to filter.

See
See forward for parameter details

template <typename T>
DELLve::Benchmark DELLve::Pooling::forward(int w, int h, int c, int n, int winH, int winW, int padH, int padW, int vStride, int hStride, std::string mode)

CuDNN Pooling Forward

Build 4D tensors using NCHW provided. Fill the input with random data using the cuRAND library. Then, calculate the forward output dimensions and set up a 4d tensor for output. Finally, return the function to run the operation with forward propagation.

Parameters
  • w: - Width of each feature map
  • h: - Height of each feature map
  • c: - Number of feature maps per image
  • n: - Number of feature maps
  • winH: - Height of pooling window
  • winW: - Width of pooling window
  • padH: - Height of the zero padding
  • padW: - Width of the zero padding
  • vStride: - Pooling vertical stride
  • hStride: - Pooling horizontal stride
  • mode: - Pooling mode ot run. Can be max, avgpad, avgnopad

template <typename T>
DELLve::Benchmark DELLve::Pooling::backward(int w, int h, int c, int n, int winH, int winW, int padH, int padW, int hStride, int wStride, std::string mode)

CuDNN Pooling Backward

Build 4D tensors using NCHW provided. Fill the input with random data using the cuRAND library. Then, calculate the backward output dimensions and set up a 4d tensor for output. Finally, return the function to run the operation with backward propagation.

See
See forward for parameter details

API Reference

Contributing

This section is coming soon. Please check back later.

Licensing

Dellve CuDNN is licensed to you under the MIT License:

Important

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.