/tensorlayer

TensorLayer: Deep Learning and Reinforcement Learning Library for Researcher and Engineer.

Primary LanguagePythonOtherNOASSERTION

Gitter

TensorLayer is a deep learning and reinforcement learning library based on Google TensorFlow. It provides rich data pre-processing, training, post-processing and serving modules that help researchers and engineers in building complex machine learning workflows.

What's New

Design Philosophy

As deep learning researchers and engineers, we have been looking for a library that can serve for various scenarios. This library shall be easy for beginners by providing mass tutorials for diverse neural networks along with applications. Later, they shall be allowed to use the same library in solving actual problems by adopting native TensorFlow APIs in sophisticated algorithms. In the end, the same library can be used again for production deployment that may has strict requirements for performance.

TensorLayer is designed for beginning, intermediate and professional deep learning users. Its architecture is largely inspired by the UNIX Philosophy :

  • Simplicity : TensorLayer lifts the low-level dataflow interface of TensorFlow to high-level deep learning modules. These modules come with detailed examples that can be deployed in minutes. A user may find it easy to bootstrap with TensorLayer, and then dive into module implementation if need.
  • Composability : If possible, deep learning modules should be composed, not built. By offering connectors to TF-Slim and Keras, TensorLayer can be used to glue existing pieces together. This yields a much better time to develop ideas and allows easy module plug-in.
  • Flexibility : A deep learning workflow can require many careful tunings. TensorLayer provides the access to the native APIs of TensorFlow and therefore help users to achieve a flexible control within the engine.
  • Performance : TensorLayer provides zero-cost compatibility for TensorFlow. It can easily run on heterogeneous platforms or multiple servers while offering native TensorFlow performance.

Why TensorLayer

A frequent question regarding TensorLayer is that why don't we use libraries like Keras and Tflearn. These libraries are comfortable to start with. They provide imperative abstractions to lower adoption barrier; but in turn mask the underlying engine from users. Though good for bootstrap, it becomes hard to tune and modify from the bottom, which is quite necessary in tackling many real-world problems.

Without compromise in simplicity, TensorLayer advocates a more flexible and composable paradigm: neural network libraries shall be used interchangeably with the native engine. This allows users to enjoy the ease of pre-built modules without losing visibility to the deep. This noninvasive nature also makes it viable to consolidate with other TF's wrappers such as TF-Slim and Keras. However, flexibility does not sacrifice performance. TensorLayer allows seamless distributed and heterogeneous deployment.

TensorLayer is in an active development stage and has received numerous contributions from an open community. It has been widely used by researchers from Imperial College London, Carnegie Mellon University, Stanford University, Tsinghua University, UCLA, Linköping University and etc., as well as engineers from Google, Microsoft, Alibaba, Tencent, ReFULE4, Bloomberg and many others. We are excited to hear about your thoughts and anticipate collaborations to promote its future. :)

Installation

TensorLayer has install prerequisites including TensorFlow, numpy and matplotlib. For GPU support, CUDA and cuDNN are required. Please check here for detailed instructions.

If you already had the pre-requisites ready (numpy, scipy, scikit-image, matplotlib and nltk(optional)), the simplest way to install TensorLayer in your python program is:

[for master version] pip install git+https://github.com/zsdonghao/tensorlayer.git (Highly Recommended)
[for stable version] pip install tensorlayer

Documentation

The documentation [Online] [PDF] [Epub] [HTML] describes the usages of TensorLayer APIs. It is also a self-contained document that walks through different types of deep neural networks, reinforcement learning and their applications in Natural Language Processing (NLP) problems.

We have included the corresponding modularized implementations of Google TensorFlow Deep Learning tutorial, so you can read the TensorFlow tutorial [en] [cn] along with our document.

Chinese documentation is also available.

Your First Program

The first program trains a multi-layer perception network to solve the MNIST problem. We use the well-known scikit-style functions such as fit() and test(). The program is self-explained.

import tensorflow as tf
import tensorlayer as tl

sess = tf.InteractiveSession()

# Prepare data
X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1,784))

# Define placeholder
x = tf.placeholder(tf.float32, shape=[None, 784], name='x')
y_ = tf.placeholder(tf.int64, shape=[None, ], name='y_')

# Define the neural network structure
network = tl.layers.InputLayer(x, name='input')
network = tl.layers.DropoutLayer(network, keep=0.8, name='drop1')
network = tl.layers.DenseLayer(network, 800, tf.nn.relu, name='relu1')
network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2')
network = tl.layers.DenseLayer(network, 800, tf.nn.relu, name='relu2')
network = tl.layers.DropoutLayer(network, keep=0.5, name='drop3')

# The softmax is implemented internally in tl.cost.cross_entropy(y, y_) to
# speed up computation, so we use identity here.
# see tf.nn.sparse_softmax_cross_entropy_with_logits()
network = tl.layers.DenseLayer(network, n_units=10, act=tf.identity, name='output')
                                
# Define cost function and metric.
y = network.outputs
cost = tl.cost.cross_entropy(y, y_, 'cost')
correct_prediction = tf.equal(tf.argmax(y, 1), y_)
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
y_op = tf.argmax(tf.nn.softmax(y), 1)

# Define the optimizer
train_params = network.all_params
train_op = tf.train.AdamOptimizer(learning_rate=0.0001).minimize(cost, var_list=train_params)

# Initialize all variables in the session
tl.layers.initialize_global_variables(sess)

# Print network information
network.print_params()
network.print_layers()

# Train the network, we recommend to use tl.iterate.minibatches()
tl.utils.fit(sess, network, train_op, cost, X_train, y_train, x, y_,
            acc=acc, batch_size=500, n_epoch=500, print_freq=5,
            X_val=X_val, y_val=y_val, eval_train=False)

# Evaluation
tl.utils.test(sess, network, acc, X_test, y_test, x, y_, batch_size=None, cost=cost)

# Save the network to .npz file
tl.files.save_npz(network.all_params , name='model.npz')

sess.close()

We provide many helper functions (like fit() , test()) that is similar to Keras to facilitate your development; however, if you want to obtain a fine-grain control over the model or its training process, you can use TensorFlow’s methods like sess.run() in your program directly (tutorial_mnist.py provides more details about this). Many more DL and RL examples can be found here.

Tricks to use TL is also a good introduction to use TensorLayer.

Examples

Examples can be found in this repository and TensorLayer Topic.

Basics

  • Multi-layer perceptron (MNIST). A multi-layer perceptron implementation for MNIST classification task, see tutorial_mnist_simple.py.
  • Multi-layer perceptron (MNIST) classification using Iterator, see method1 and method2.

Computer Vision

Natural Language Processing

Adversarial Learning

Reinforcement Learning

Applications

Special Examples

Notes

License

TensorLayer is released under the Apache 2.0 license.

Contributions

TensorLayer is maintained by numerous Github contributors here.

Citation

If you find it is useful, please cite our paper in your project and paper.

@article{TensorLayer2017,
author = {Dong, Hao and Supratak, Akara and Mai, Luo and Liu, Fangde and Oehmichen, Axel and Yu, Simiao and Guo, Yike},
journal = {ACM Multimedia},
title = {{TensorLayer: A Versatile Library for Efficient Deep Learning Development}},
url = {http://tensorlayer.org},
year = {2017}
}