This repo aims to benchmark RedisAI against everything else which includes
- RedisAI with Tensorflow as backend vs Tensorflow python runtime
- RedisAI with PyTorch as backend vs PyTorch python runtime
- RedisAI vs GRPC servers (Tensorflow Serving & custom GRPC server for PyTorch and ONNX)
- RedisAI vs Flask servers
- RedisAI with ONNX as backend vs ONNXRuntime (upcoming)
- RedisAI vs MxNet Model Server (upcoming)
- RedisAI vs TensorRT (upcoming)
Models used in for the benchmark can be found here and the docker images can be found in the docker hub
- OS Platform and Distribution: Linux Ubuntu 18.04
- Device: CPU
- Python version: 3.6
- Tensorflow version: 1.12.0
- TensorFlow optimizations: ON
(grpc for tensorflow is TFServing)
For running the benchamarks (right now it runs with Tensorflow which is built with optimizations ON, if your hardware doesn't support it, keen an eye on this repo, we'll update it soon with prebuilt binary of TF available in pypy), cd to the repo and run
python run.pyThis will pull the required docker images, bring it up when requires and run the clients for each. Available command line configurations are
usage: run.py [-h] [--device {cpu,gpu,all}]
[--backend {tensorflow,pytorch,all}] [--count COUNT] [--exp ...]
optional arguments:
-h, --help show this help message and exit
--device {cpu,gpu,all}
Run benchmarking for CPU or GPU or both
--backend {tensorflow,pytorch,all}
Run benchmarking for tensorflow or pytoch or onnx
--count COUNT How many iterations to take average from, for each
experiment
--exp ...
--exp is for specifying experiments which are currently native, flask, redisai, grpc. Make sure --exp is specified at the end of your command. An example command to run only redisai and flask of pytorch, only two times is below
python run.py --backend pytorch --count 2 --exp flask redisai