Extensor

Extensor implements Tensorflow bindings for inference in Elixir. This library can be used to execute computation graphs created in Tensorflow on the CPU or GPU. Extensor provides minimal abstractions over the Tensorflow C library and includes as little custom native code as possible. These NIFs have been extensively tested for memory leaks and paralellism safety so that the library can be relied on for production use.

Status

The API reference is available here.

Installation

Hex

def deps do
  [
    {:extensor, "~> 2.3"}
  ]
end

Dependencies

This project requires the Tensorflow C headers/libraries. For development, these can be installed by following the official Tensorflow instructions.

For docker deployment, see the sample dockerfiles in the docker directory. Docker for ubuntu can be tested with the following commands.

docker build -t extensor -f docker/ubuntu-cpu.dockerfile .
docker run --rm -it extensor mix test

If you have nvidia tools installed, you can test on the GPU by using the ubuntu-gpu.dockerfile and substituting nvidia-docker for docker above.

Usage

For a simple example, Extensor can be used to evaluate the Pythagorean identity (c² = a² + b²). The following python script can be used to create and save a graph that calculates the length of the hypotenuse.

import tensorflow as tf

a = tf.placeholder(tf.float32, name='a')
b = tf.placeholder(tf.float32, name='b')
c = tf.sqrt(tf.add(tf.square(a), tf.square(b)), name='c')

with tf.Session() as session:
    tf.train.write_graph(session.graph_def,
                         'test/data',
                         'pythagoras.pb',
                         as_text=False)

This model can then be used in Elixir to evaluate the compute graph and calculate a few hypotenuses. This model file is also available in the repo under test/data/pythagoras.pb.

session = Extensor.Session.load_frozen_graph!("test/data/pythagoras.pb")

input = %{
  "a" => Extensor.Tensor.from_list([3, 5]),
  "b" => Extensor.Tensor.from_list([4, 12])
}

output = Extensor.Session.run!(session, input, ["c"])

Extensor.Tensor.to_list(output["c"])

This block should output the list [5.0, 13.0], which corresponds to the lengths of the hypotenuses of the first two Pythagorean triples.

Model Formats

Extensor supports the frozen graph_def and saved_model serialization formats.

For example, the Google Inception model has its weights frozen to constant tensors, so that it can be loaded directly from a protobuf via load_frozen_graph.

However, the frozen graph approach may not work for models that contain unfreezable variables (like RNNs). For these models, Extensor supports the Tensorflow saved_model format, which is the format used by Tensorflow serving (TFS). The saved_model format is loaded from a directory path, which includes model metadata and initial variable weights. These models can be loaded with load_saved_model.

Configuration

Extensor supports passing a ConfigProto object when creating a session for inference configuration. See the Tensorflow.ConfigProto module for more information on the configuration data structures.

config = %{
    Tensorflow.ConfigProto.new()
    | gpu_options: %{
        Tensorflow.GPUOptions.new()
        | allow_growth: true
    }
}

session = Extensor.Session.load_saved_model!("test/data/pythagoras", config)

Tensor Format

The previous examples used the tensor from_list/to_list convenience functions to read/write tensor data. However, any binary tensor data may be used, as long as it adheres to Tensorflow's layout and endianness (native) conventions.

In general, a tensor is defined by type, shape, and data parameters. These can be used to construct an Extensor.Tensor struct directly.

The data type is an atom identifying one of the supported tensorflow data types.

atom	tensorflow type
`:float`	`TF_FLOAT`
`:double`	`TF_DOUBLE`
`:int32`	`TF_INT32`
`:uint8`	`TF_UINT8`
`:int16`	`TF_INT16`
`:int8`	`TF_INT8`
`:string`	`TF_STRING`
`:complex64`	`TF_COMPLEX64`
`:complex`	`TF_COMPLEX`
`:int64`	`TF_INT64`
`:bool`	`TF_BOOL`
`:qint8`	`TF_QINT8`
`:quint8`	`TF_QUINT8`
`:qint32`	`TF_QINT32`
`:bfloat16`	`TF_BFLOAT16`
`:qint16`	`TF_QINT16`
`:quint16`	`TF_QUINT16`
`:uint16`	`TF_UINT16`
`:complex128`	`TF_COMPLEX128`
`:half`	`TF_HALF`
`:resource`	`TF_RESOURCE`
`:variant`	`TF_VARIANT`
`:uint32`	`TF_UINT32`
`:uint64`	`TF_UINT64`

The tensor shape is a tuple containing the dimensions of the tensor. The data field contains the binary tensor data, whose size must be consistent with the tensor shape. The following is an example of a tensor struct.

%Tensor{
    type: :double,
    shape: {2, 1},
    data: <<1::native-float-64, 2::native-float-64>>
}

Matrex Integration

Extensor supports optional integration with Matrex. To demonstrate, we'll re-use our Pythagoras example test/data/pythagoras.pb

a = Matrex.new([[3, 5], [7, 9]])
b = Matrex.new([[4, 12], [24, 40]])

input = %{
  "a" => Extensor.Matrex.to_tensor(a),
  "b" => Extensor.Matrex.to_tensor(b)
}

session = Extensor.Session.load_frozen_graph!("test/data/pythagoras.pb")
output = Extensor.Session.run!(session, input, ["c"])

output |> Map.get("c") |> Extensor.Matrex.from_tensor()

This block should output a 2x2 matrix, which corresponds to the lengths of the hypotenuses of the first four Pythagorean triples.

Development

The Tensorflow protocol buffer wrappers were generated using the protobuf-elixir library with the following command, assuming Tensorflow is cloned in the ../tensorflow directory:

protoc \
  --elixir_out=lib \
  --proto_path=../tensorflow \
  $(ls -1 \
    ../tensorflow/tensorflow/core/framework/*.proto \
    ../tensorflow/tensorflow/core/protobuf/*.proto \
    ../tensorflow/tensorflow/stream_executor/*.proto)

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

pylon/extensor