This pet project of mine serves as an end-to-end example of how to create a privacy-first machine learning application that runs in the browser. It demonstratetes how to train a PyTorch model with fastai, export it to ONNX format and run it in the browser using onnx.js inside a react app.
Because I am a dog lover (sorry cat friends, but you can easily train your own classifier model and build an app for cat breeds!), my goal is to create a simple, yet functional and realistic react app that can classify a dog's breed using an image of dog. The app should be user friendly, so I could actually use it when I am interested in what breed a certain dog is that I encountered.
I used the Stanford Dogs Dataset, which contains 137 different breeds of dogs with about 150 images per breed.
The training process is described in detail in this jupyter notebook. Note: You can run this notebook for free in Google Colabatory if you have a Google account.
I am using a resnet18 architecture, but I am planning to investigate more efficient architectures in the future and will update this project accordingly if I find them to be useful.
The fairly simple react app was set up using create-react-app, see cra-instructions.md for the generated README file. On top of react, I am using Material-UI, a very powerful react imlementation of the material design language.
To load and run the model, I am using onnx.js by Microsoft, which can load ONNX formatted machine learning models in node or the browser.
Therer is no backend! Because the model runs in the browser, you don't need a server that runs the model and you preserve your users privacy. You can have your 🎂 and eat it too!