Fabrik: A Python repository from Jatana

Fabrik is an online collaborative platform to build, visualize and train deep learning models via a simple drag-and-drop interface. It allows researchers to collaboratively develop and debug models using a web GUI that supports importing, editing and exporting networks written in widely popular frameworks like Caffe, Keras, and TensorFlow.

This app is presently under active development and we welcome contributions. Please check out our issues thread to find things to work on, or ping us on Gitter.

Installation Instructions

Setting up Fabrik on your local machine is really easy. You can setup Fabrik using two methods:

Using Docker

Get the source code on to your machine via git.

git clone https://github.com/Cloud-CV/Fabrik.git && cd Fabrik

Rename settings/dev.sample.py as dev.py.

cp settings/dev.sample.py settings/dev.py

Build and run the Docker containers. This might take a while. You should be able to access Fabrik at 0.0.0.0:8000.
```
docker-compose up --build
```

Setup Authenticaton for Docker Environment

Go to Github Developer Applications and create a new application. here
For local deployments the following is what should be used in the options:
- Application name: Fabrik
- Homepage URL: http://0.0.0.0:8000
- Application description: Fabrik
- Authorization callback URL: http://0.0.0.0:8000/accounts/github/login/callback/
Github will provide you with a Client ID and Secret Key, save these.
Create a superuser in django service of docker container
```
docker-compose run django python manage.py createsuperuser
```
Note: Before creating make sure that django service of docker image is running, it can be done by executing docker-compose up followed by Ctrl + C to save docker configuration.
Open http://0.0.0.0:8000/admin and login with credentials from step 4.
Setting up Social Accounts in django admin
- Under the Social Accounts tab open Socialapplications, click on Add Social Application.
- Choose the Provider of social application as Github & name it Github.
- Add the sites available to the right side, so github is allowed for the current site.
- Copy and paste your Client ID and Secret Key into the apppropriate fields and Save.
Go to Sites tab and update the Domain name to 0.0.0.0:8000.

Using Virtual Environment

First set up a virtualenv

sudo apt-get install python-pip python-dev python-virtualenv
virtualenv --system-site-packages ~/Fabrik
source ~/Fabrik/bin/activate

Clone the repository

git clone --recursive https://github.com/Cloud-CV/Fabrik.git

Rename settings/dev.sample.py as settings/dev.py and change credential in settings/dev.py
```
cp settings/dev.sample.py settings/dev.py
```
Replace the hostname to localhost in settings/dev.py line 14.
Install redis server and replace the hostname to localhost in settings/common.py line 99.
```
sudo apt-get install redis-server
```
Replace celery result backend in settings/common.py line 122 with
```
CELERY_RESULT_BACKEND = 'redis://redis:6379/0'
```
Replace celery broker url and result backend hostname to localhost in ide/celery_app.py line 8 with
```
broker='redis://redis:6379/0'
backend='redis://redis:6379/0'
```
If you have Caffe, Keras and Tensorflow already installed on your computer, skip this step
- For Linux users
```
cd Fabrik/requirements
yes Y | sh caffe_tensorflow_keras_install.sh
```
  Open your ~/.bashrc file and append this line at the end
```
export PYTHONPATH=~/caffe/caffe/python:$PYTHONPATH
```
  Save, exit and then run
```
source ~/.bash_profile
cd ..
```
- For Mac users
Install dependencies

For developers:
```
pip install -r requirements/dev.txt
```
Others:
```
pip install -r requirements/common.txt
```

Install postgres >= 9.5

Setup postgres database

Start postgresql by typing sudo service postgresql start
Now login as user postgres by running sudo -u postgres psql and type the commands below

  CREATE DATABASE fabrik;
  CREATE USER admin WITH PASSWORD 'fabrik';
  ALTER ROLE admin SET client_encoding TO 'utf8';
  ALTER ROLE admin SET default_transaction_isolation TO 'read committed';
  ALTER ROLE admin SET timezone TO 'UTC';
  ALTER USER admin CREATEDB;

Exit psql by typing in \q and hitting enter.

Migrate


python manage.py makemigrations caffe_app
python manage.py migrate

Install node modules

npm install
npm install --save-dev json-loader
sudo npm install -g webpack
webpack --progress --watch --colors

To start celery worker

celery -A ide worker --app=ide.celery_app  --loglevel=info

Setup Authenticaton for Virtual Environment

Go to Github Developer Applications and create a new application. here
For local deployments the following is what should be used in the options:
- Application name: Fabrik
- Homepage URL: http://localhost:8000
- Application description: Fabrik
- Authorization callback URL: http://localhost:8000/accounts/github/login/callback/
Github will provide you with a client ID and secret, save these.
Create a superuser in django
```
python manage.py createsuperuser
```
Start the application
```
python manage.py runserver
```
Open http://localhost:8000/admin
Login with credentials from step
Setting up Social Accounts in django admin
- Under the Social Accounts tab open Socialapplications, click on Add Social Application.
- Choose the Provider of social application as Github & name it Github.
- Add the sites available to the right side, so github is allowed for the current site.
- Copy and paste your Client ID and Secret Key into the apppropriate fields and Save.
Go to Sites tab and update the Domain name to localhost:8000.

Note: For testing, you will only need one authentication backend. However, if you want to try out Google's authentication then, you will need to follow the same steps as above, but switch out the Github for google.

Usage

python manage.py runserver

Example

Use example/tensorflow/GoogleNet.pbtxt for tensorflow import
Use example/caffe/GoogleNet.prototxt for caffe import
Use example/keras/vgg16.json for keras import

Tested models

The model conversion between currently supported frameworks is tested on some models.

Models	Caffe	Keras	Tensorflow
Inception V3	√	√	√
Inception V4	√	√	√
ResNet 101	√	√	√
VGG 16	√	√	√
GoogLeNet	√	×	×
SqueezeNet	√	×	×
DenseNet	√	×	×
AllCNN	√	×	×
AlexNet	√	√	√
FCN32 Pascal	√	×	×
YoloNet	√	√	√
Pix2Pix	√	×	×
VQA	√	√	√

Note: For models that use a custom LRN layer (Alexnet), Keras expects the custom layer to be passed when it is loaded from json. LRN.py is located in keras_app/custom_layers. Alexnet import for Keras

Documentation

License

This software is licensed under GNU GPLv3. Please see the included License file. All external libraries, if modified, will be mentioned below explicitly.

Jatana/Fabrik