Bender

Bender is an abstraction layer over MetalPerformanceShaders useful for working with neural networks.

Introduction
Why did we need Bender
Basic usage
Requirements
Getting involved
Examples
Installation
Changelog

The documentation can be found under the Documentation folder:

API contains the most important information to get started.
Supported Layers explains which layers are supported and how they map to TensorFlow ops.
Importing explains how to import models from other frameworks such as TensorFlow. You can also find information on how to enhance this functionality for custom implementations.

Introduction

Bender is an abstraction layer over MetalPerformanceShaders which is used to work with neural networks. It is of growing interest in the AI environment to execute neural networks on mobile devices even if the training process has been done previously. We want to make it easier for everyone to execute pretrained networks on iOS.

Bender allows you to easily define and run neural networks using the most common layers like Convolution, Pooling, FullyConnected and some normalizations among others. It is also flexible in the way it receives the parameters for these layers.

We also want to support loading models trained on other frameworks such as TensorFlow or Caffe2. Currently Bender includes an adapter for TensorFlow that loads a graph with variables and "translates" it to Bender layers. This feature supports a subset of TensorFlow's operations but we plan to enhance it to cover more cases.

Why did we need Bender?

At Xmartlabs we were about to start a Machine Learning project and investigated frameworks to use in iOS. We found MetalPerformanceShaders useful but not very user friendly and we saw ourselves repeating a lot of code and information. That is why we starting building a framework to handle that kind of stuff.

We also found ourselves creating scripts to translate the models we had from training with TensorFlow to iOS. This means transposing the weights to the MPSCNN format and also mapping the parameters of the different kinds of layers in TensorFlow to the parameters used by the MPSCNN kernels. TensorFlow can be compiled for iOS but currently it does not support running on GPU which we wanted to do. We also did not want to include TensorFlow's static library into our project. This is why we also started to work on an adapter that would parse a TF graph and translate it to our Bender layers.

Usage

You can define your own network in Bender using our custom operator or you can load a model exported from TensorFlow. Defining a network and loading a model can be done like this:

import MetalBender

let url = Bundle.main.url(forResource: "myModel", withExtension: "pb")! // A TensorFlow model.
let network = Network.load(url: url, inputSize: LayerSize(h: 256, w: 256, f: 3))

network.run(input: /* ... */) { output in
    // ...
}

You can read more information about this in Importing.

If you want to define your network yourself you can do it like this:

let network = Network(inputSize: LayerSize(h: 256, w: 256, f: 3))

network.start
    ->> Convolution(convSize: ConvSize(outputChannels: 16, kernelSize: 3, stride: 2))
    ->> InstanceNorm()
    ->> Convolution(convSize: ConvSize(outputChannels: 32, kernelSize: 3, stride: 2), neuronType: .relu)
    ->> InstanceNorm()
    ->> FullyConnected(neurons: 128)
    ->> Neuron(type: .tanh)
    ->> FullyConnected(neurons: 10)
    ->> Softmax()
// ...

and once you're done with all your layers:

network.initialize()

To know more about this have a look at API.

Requirements

Xcode 9
iOS 11.0+ (but deployment target is iOS 10.0, so iOS 10 is supported)

Getting involved

If you want to contribute please feel free to submit pull requests.
If you have a feature request please open an issue.
If you found a bug or need help please check older issues, FAQ and threads on StackOverflow before submitting an issue.

Before contribute check the CONTRIBUTING file for more info.

If you use Bender in your app We would love to hear about it! Drop us a line on Twitter.

Examples

Follow these steps to run the examples:

Clone Bender repository (or download it).
Run carthage update --platform iOS in the downloaded folder.
Open Bender workspace and run the Example project.

There is an Image recognition example which includes a MobileNet model in Bender and one in CoreML. It is also set up to run an Inception model but you will have to download it separately as it is almost 100 MB in size. You can download it from http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz but then you have to freeze it and add it to the 'Example' Xcode project as 'inception_v3.pb'.

Installation

CocoaPods

To install Bender, simply add the following line to your Podfile:

pod 'MetalBender', '~> 0.5'

Remember that Bender compiles for iOS 10. So you must add platform :ios, '10.0' to your Podfile

Carthage

Carthage is a simple, decentralized dependency manager for Cocoa.

To install Bender, add the following line to your Cartfile:

github "xmartlabs/Bender" ~> 0.5

Then run:

carthage update --platform iOS

Finally, drag the built .framework binaries for MetalBender, MetalPerformanceShadersProxy and SwiftProtobuf to your application's Xcode project.

Author

Xmartlabs SRL (@xmartlabs)

Change Log

This can be found in the CHANGELOG.md file.

License

Citation

If you use this code in your research please cite us:

@misc{xmartlabs-2017-bender,
  author = {Mathias Claassen and Santiago Castro},
  title = {Bender: Easily craft fast Neural Networks on {iOS}!},
  year = {2017},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://xmartlabs.github.io/Bender/}}
}

xmartlabs/Bender