/GpuDLBenchmark

GPU benchmark on deep learning

Primary LanguagePython

Overview

A framework to benchmark common deep-learning frameworks. Easy to use yet fully customizable.

Usage

1. Set up your config file

Create a config file to specify running details. A sample config file is already provided under the root path of the project:example_config.csv

It's a csv-format file. You can copy this file and change the content at your will.

framework,network_type,network_name,device_id,device_count,cpu_count,batch_size,number_of_epochs,epoch_size,learning_rate,synthetic,enabled
tensorflow,cnn,alexnet,0,1,0,64,40,50000,0.01,0,1
... <more rows>

Explanation of csv head row fields:

  • framework

    Deep learning framework used to perform benchamark. Only support Tensorflow so far.

  • network_type

    Deep learning network type. Support cnn/rnn/fc corresponding to CNN/RNN/FCN network type.

  • network_name

    Deep learning network name. Support:

    1. cnn: alexnet, resnet

    2. rnn: lstm

    3. fcn: fcn5

  • device_id

    Specify which GPU(s) to use. Use semicolon as a seperator if more than one GPU is used.

  • device_count

    number of GPU(s) used.

  • cpu_count

    number of CPU core used.

    Attention: cpu_count with a value 0 means using all CPU cores.

  • batch_size

    The number of training examples in one forward/backward pass.

  • number_of_epochs

    One epoch means one forward pass and one backward pass of all the training examples.

  • epoch_size

    Size of the trainning set.

  • learning_rate

    Learning rate in deep learning.

  • synthetic

    Whether use synthetic data or not. 1 for true, use 0 otherwise.

  • enabled

    Toggle a config row. 0 will not be executed.

2. Start the benchmark

Simply run the following command to start benchmarking:

python -config your_config.csv -log_dir DirPathToSaveLogs -test_summary_file FilePathToSaveReport

As the name says,

  • config filepath of your config csv file
  • log_dir directory to save intermediate logs
  • test_summary_file filepath to save the benchmark results (in csv format)

Prerequisites

You should have the corresponding deep learning frameworks installed (e.g. TensorFlow, Coffee, etc.).

Also all cuda-related things (GPU driver, cuDNN library, etc.) is already set up.

Finally specify the CIFAR10 and MNIST dataset path in globalconfig.py.

You should create an image or use docker to simplify your setup work across different test machines.