BillPei/tensorpack

A Neural Net Training Interface on TensorFlow

tensorpack

A neural net training interface based on TensorFlow.

See some examples to learn about the framework:

Vision:

• Multi-GPU training of ResNet on ImageNet
• Generative Adversarial Network(GAN) variants, including DCGAN, InfoGAN, Conditional GAN, WGAN, BEGAN, DiscoGAN, Image to Image, CycleGAN.
• DoReFa-Net: train binary / low-bitwidth CNN on ImageNet
• Fully-convolutional Network for Holistically-Nested Edge Detection(HED)
• Spatial Transformer Networks on MNIST addition
• Visualize CNN saliency maps
• Similarity learning on MNIST

Reinforcement Learning:

• Deep Q-Network(DQN) variants on Atari games, including DQN, DoubleDQN, DuelingDQN.
• Asynchronous Advantage Actor-Critic(A3C) with demos on OpenAI Gym

Speech / NLP:

• LSTM-CTC for speech recognition
• char-rnn for fun
• LSTM language model on PennTreebank

Examples are not only for demonstration of the framework -- you can train them and reproduce the results in papers.

Features:

It's Yet Another TF wrapper, but different in:

1. It's not a model wrapper.

   • There are already too many symbolic function wrappers. Tensorpack includes only a few common models, but you can use any other model wrappers within tensorpack, such as sonnet/Keras/slim/tflearn/tensorlayer/....

2. Focus on training speed.

   • Speed comes for free with tensorpack -- it uses TensorFlow in the correct way. Even on a tiny CNN example, the training runs 1.6x faster than the equivalent Keras code.

   • Data-parallel multi-GPU training is off-the-shelf to use. It is as fast as Google's official benchmark.

   • Data-parallel distributed training is off-the-shelf to use. It is as slow as Google's official benchmark.

3. Focus on large datasets.

   • It's painful to read/preprocess data through TF. tensorpack helps you load large datasets (e.g. ImageNet) in pure Python with autoparallelization. It also naturally works with TF Queues or tf.data.

4. Interface of extensible Callbacks. Write a callback to implement everything you want to do apart from the training iterations, and enable it with one line of code. Common examples include:

   • Change hyperparameters during training
   • Print some tensors of interest
   • Monitor GPU utilization
   • Send error rate to your phone

See tutorials to know more about these features.

Install:

Dependencies:

• Python 2 or 3
• TensorFlow >= 1.0.0 (>=1.1.0 for Multi-GPU)
• Python bindings for OpenCV (Optional, but required by a lot of features)

pip install -U git+https://github.com/ppwwyyxx/tensorpack.git
# or add `--user` to avoid system-wide installation.

Besides, if you only want to use tensorpack.dataflow alone as a data processing library, TensorFlow is also optional.