DeepCL

Global Contents

Table of Contents generated with DocToc

OpenCL library to train deep convolutional networks

Functionalities:

convolutional layers
max-pooling
normalization layer
random translations, as in Flexible, High Performance Convolutional Neural Networks for Image Classification
random patches, as in ImageNet Classification with Deep Convolutional Networks
multinet, ie Multi-column deep convolutional network, McDnn
simple command-line network specification, as per notation in Multi-column Deep Neural Networks for Image Classification
pad-zeros possible for convolutional layer
various activation functions available:
- tanh
- scaled tanh (1.7519 * tanh(2/3x) )
- linear
- sigmoid
- relu
- softmax
fully-connected layers
various loss layers available:
- square loss
- cross-entropy
- multinomial cross-entropy (synonymous with multinomial logistic, etc)
Q-learning

Example usage:

intend to target 19 x 19 Go boards, eg something similar to Clark and Storkey or Maddison, Huang, Sutskever and Silver
- obtained 36.3% test accuracy, on next move prediction task, using 33.6 million training examples from kgsgo v2 dataset
- commandline used ./clconvolve1 dataset=kgsgoall netdef=32c5{z}-32c5{z}-32c5{z}-32c5{z}-32c5{z}-32c5{z}-500n-361n numepochs=3 learningrate=0.0001
- 3 epochs, 1.5 days per epoch, on an Amazon GPU instance, comprising half an NVidia GRID K520 GPU (about half as powerful as a GTX780)
obtained 99.5% test accuracy on MNIST, using netdef=rt2-8c5{padzeros}-mp2-16c5{padzeros}-mp3-150n-10n numepochs=20 multinet=6 learningrate=0.002
- epoch time 99.8 seconds, using an Amazon GPU instance, ie half an NVidia GRID K520 GPU (since we are learning 6 nets in parallel, so 16.6seconds per epoch per net)

For Python wrappers, please see python/README.md

Pre-built binaries are available for Windows-64, for certain releases. In order to use them you need:

Check if you have an OpenCL-enabled device on your system
- ideally a GPU, or accelerator, since there is no attempt to optimize DeepCL for CPUs (at least, not currently, could change, feel free to submit a pull request :-) )
Try running gpuinfo (from OpenCLHelper, but built as part of this project too, for ease of use )
- it should output at least one OpenCL-enabled device
- if it doesn't, then you need to make sure you have an OpenCL-enabled device, and that appropriate drivers are installed, and that the ICD is configured appropriately (registry in Windows, and /etc/OpenCL/vendors in linux)

Please raise an issue, let me know you're interested.

If it's on my list of things I was going to do sooner or later anyway (see below), I might do it sooner rather than later.
If it's to do with usability, I will try to make that a priority

cogapp generator is used extensively, to accelerate development, reduce the number of manual copy-and-pasting and so on. Specifically, it's used for:
- generating header declarations from .cpp definition files
- generating fluent-style argument classes for certain tests
- ... and more uses will surely be found :-)
You need Python installed and available for this to work. You don't need python just to build the sources, but if you do have python installed, and you flip the PYTHON_AVAILABLE switch in the cmake configuration, then a lot of manual editing will no longer be necessary :-)