This repository contains a simple web app for making predictions using the MNIST dataset. The purpose is to understand torchserve and how to combine it with a webservice. The model for making the predictions has been downloaded from https://github.com/pytorch/serve/tree/master/examples/image_classifier/mnist/mnist_cnn.pt
In the repo https://github.com/pytorch/serve/tree/master/examples/image_classifier/mnist/ an example how to serve this model using a customized handler based on the ImageClassifier. I used the BaseHandler to get familiar with it. The handler script is called mnist_handler_base.py and I used this post to write it: https://towardsdatascience.com/deploy-models-and-create-custom-handlers-in-torchserve-fc2d048fbe91. The base handler script can be found here: https://github.com/pytorch/serve/blob/master/ts/torch_handler/base_handler.py
Create the .mar
file, which contains all information to deploy the model (in order to execute this some dependencies need to be installed)
torch-model-archiver --model-name mnist --version 2.0 --model-file mnist.py --serialized-file mnist_cnn.pt --handler mnist_handler_base.py --force
Move the created file into the model-store
folder
mv mnist.mar model-store/
Use docker to serve the model
docker run --rm -it -p 8080:8080 -p 8081:8081 --name mar -v $(pwd)/model-store:/home/model-server/model-store -v $(pwd):/home/model-server/examples pytorch/torchserve:latest torchserve --start --model-store model-store --models mnist=mnist.mar
Check which models are registered:
curl http://127.0.0.1:8081/models/
output
{
"models": [
{
"modelName": "mnist",
"modelUrl": "mnist.mar"
}
]
}
Make predictions:
curl http://127.0.0.1:8080/predictions/mnist -T 3.png
, output 3
now we want to combine the above with a simple web app to make the predictions from an uploaded image:
- in folder
deployment
is a Dockerfile frompytorch/torchserve:latest
, which only copies the.mar
file to themodel-store
folder. nd starts torchserve (Note: change latest to other version)- build the image:
build -t torchserve-mar:v1
- build the image:
- In subfolder
app
- create file
app.py
and subfolderstemplates
andstatic
templates
contains html content of appstatic
is for saveig the uploaded imagesapp.py
is script that creates the app to make predictions- in folder
app
create Dockerfile to run the app - build the image
docker-build -t app:v1
- create file
- in folder
deployment
usedocker-compose.yaml
to run both services - start services with
docker-compose up