Deep-Learning-in-Production
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
Convert PyTorch Models in Production:
- PyTorch Production Level Tutorials [Fantastic]
- The road to 1.0: production ready PyTorch
- PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS [Good Article]
- Model Serving in PyTorch
- PyTorch Summer Hackathon [Very Important]
- Deploying PyTorch and Building a REST API using Flask [Important]
- PyTorch model recognizing hotdogs and not-hotdogs deployed on flask
- Serving PyTorch 1.0 Models as a Web Server in C++ [Useful Example]
- PyTorch Internals [Interesting & Useful Article]
- Flask application to support pytorch model prediction
- Serving PyTorch Model on Flask Thread-Safety
- Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX
- Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX (Another Version)
- EuclidesDB - multi-model machine learning feature database with PyTorch
- EuclidesDB - GitHub
- WebDNN: Fastest DNN Execution Framework on Web Browser
- FastAI PyTorch Serverless API (with AWS Lambda)
- FastAI PyTorch in Production (discussion)
Convert PyTorch Models to C++:
- Loading a PyTorch Model in C++ [Fantastic]
- PyTorch C++ API [Bravo]
- An Introduction To Torch (Pytorch) C++ Front-End [Very Good]
- Blogs on using PyTorch C++ API [Good]
- ATen: A TENsor library
- Important Issue about PyTorch-like C++ interface
- PyTorch C++ API Test
- PyTorch via C++ [Useful Notes]
- AUTOGRADPP
- PyTorch C++ Library
- Direct C++ Interface to PyTorch
- A Python module for compiling PyTorch graphs to C
Deploy TensorFlow Models in Production:
- How to deploy Machine Learning models with TensorFlow - Part1
- How to deploy Machine Learning models with TensorFlow - Part2
- How to deploy Machine Learning models with TensorFlow - Part3
- Neural Structured Learning (NSL) in TensorFlow [Great]
- Building Robust Production-Ready Deep Learning Vision Models
- Creating REST API for TensorFlow models
- "How to Deploy a Tensorflow Model in Production" by Siraj Raval on YouTube
- Code for the "How to Deploy a Tensorflow Model in Production" by Siraj Raval on YouTube
- How to deploy an Object Detection Model with TensorFlow serving [Very Good Tutorial]
- Freeze Tensorflow models and serve on web [Very Good Tutorial]
- How to deploy TensorFlow models to production using TF Serving [Good]
- How Zendesk Serves TensorFlow Models in Production
- TensorFlow Serving Example Projects
- Serving Models in Production with TensorFlow Serving [TensorFlow Dev Summit 2017 Video]
- Building TensorFlow as a Standalone Project
- TensorFlow C++ API Example
- TensorFlow.js
- Introducing TensorFlow.js: Machine Learning in Javascript
Convert Keras Models in Production:
- Deep learning in production with Keras, Redis, Flask, and Apache [Rank: 1st & General Usefult Tutorial]
- Deploying a Keras Deep Learning Model as a Web Application in Python [Very Good]
- Deploying a Python Web App on AWS [Very Good]
- Deploying your Keras model
- Deploying your Keras model using Keras.JS
- "How to Deploy a Keras Model to Production" by Siraj Raval on Youtube
- Deploy Keras Model with Flask as Web App in 10 Minutes [Good Repository]
- Deploying Keras Deep Learning Models with Flask
- keras2cpp
Deploy MXNet Models in Production:
- Model Server for Apache MXNet
- Running the Model Server
- Exporting Models for Use with MMS
- Single Shot Multi Object Detection Inference Service
- Amazon SageMaker
- How can we serve MXNet models built with gluon api
- MXNet C++ Package
- MXNet C++ Package Examples
- MXNet Image Classification Example of C++
- MXNet C++ Tutorial
- An introduction to the MXNet API [Very Good Tutorial for Learning MXNet]
- GluonCV
- GluonNLP
- Model Quantization for Production-Level Neural Network Inference [Excellent]
General Deep Learning Compiler Stack:
Model Conversion between Deep Learning Frameworks:
- ONNX (Open Neural Network Exchange)
- Tutorials for using ONNX
- MMdnn [Fantastic]
- Convert Full ImageNet Pre-trained Model from MXNet to PyTorch [Fantastic, & Full ImageNet model means the model trained on ~ 14M images]
Some Caffe2 Tutorials:
- Mnist using caffe2
- Caffe2 C++ Tutorials and Examples
- Make Transfer Learning of SqueezeNet on Caffe2
- Build Basic program by using Caffe2 framework in C++
Some Useful Resources for Designing UI (Front-End Development):
- ReactJS vs Angular5 vs Vue.js
- A comparison between Angular and React and their core languages
- A Guide to Becoming a Full-Stack Developer [Very Good Tutorial]
- Roadmap to becoming a web developer in 2018 [Very Good Repository]
- Modern Frontend Developer in 2018
- Roadmap to becoming a React developer in 2018
- 2019 UI and UX Design Trends [Good]
- Web Developer Monthly
- 23 Best React UI Component Frameworks
- 9 React Styled-Components UI Libraries for 2018
- 35 New Tools for UI Design
- 5 Tools To Speed Up Your App Development [Very Good]
- How to use ReactJS with Webpack 4, Babel 7, and Material Design
- Adobe Typekit [Great fonts, where you need them]
- Build A Real World Beautiful Web APP with Angular 6
- You Don't Know JS
- JavaScript Top 10 Articles
- Web Design with Adobe XD
- INSPINIA Bootstrap Web Theme
- A Learning Tracker for Front-End Developers
- The best front-end hacking cheatsheets — all in one place [Useful & Interesting]
- GUI-fying the Machine Learning Workflow (Machine Flow)
- Electron - Build cross platform desktop apps with JavaScript [Very Good]
Mobile Development:
- PyTorch Mobile [Excellent]
- Mobile UI Design Trends In 2018
- ncnn - high-performance neural network inference framework optimized for the mobile platform [Useful]
- Alibaba - MNN
- Awesome Mobile Machine Learning
- EMDL - Embedded and Mobile Deep Learning
- Fritz - machine learning platform for iOS and Android
- TensorFlow Lite
Back-End Development Part:
- Modern Backend Developer in 2018
- Deploying frontend applications — the fun way [Very Good]
- RabbitMQ [Message Broker Software]
- Celery [Distributed Task Queue]
- Kafka [Distributed Streaming Platform]
- Kubernetes - GitHub
- Jenkins and Kubernetes with Docker Desktop
- Create Cluster using docker swarm
- PySyft - A library for encrypted, privacy preserving deep learning
GPU Management Libraries:
- GPUtil
- py3nvml [Python 3 binding to the NVIDIA Management Library]
- PyCUDA - GitHub
- PyCUDA
- PyCUDA Tutorial
- setGPU
- Monitor your GPUs [Excellent]
Speed-up & Scalabale Python Codes:
- Numba - makes Python code fast
- Dask - natively scales Python
- Neural Network Distiller [Distillation & Quantization of Deep Learning Models in PyTorch]
- PocketFlow - An Automatic Model Compression (AutoMC) framework [Great]
- Introducing the Model Optimization Toolkit for TensorFlow
- TensorFlow Model Optimization Toolkit — Post-Training Integer Quantization
- TensorFlow Post-training Quantization
- Dynamic Quantization in PyTorch
- Static Quantization in PyTorch
- NVIDIA DALI - highly optimized data pre-processing in deep learning
- Horovod - Distributed training framework
- ONNX Float32 to Float16
Other:
- Facebook Says Developers Will Love PyTorch 1.0
- Some PyTorch Workflow Changes
- PyTorch and Caffe2 repos getting closer together
- PyTorch or TensorFlow?
- Choosing a Deep Learning Framework in 2018: Tensorflow or Pytorch?
- Deep Learning War between PyTorch & TensorFlow
- Embedding Machine Learning Models to Web Apps (Part-1)
- Deploying deep learning models: Part 1 an overview
- Machine Learning in Production
- how you can get a 2–6x speed-up on your data pre-processing with Python
- Making your C library callable from Python
- MIL WebDNN
- Multi-GPU Framework Comparisons [Great]