ML repository for Nutrifit project. The ML part of Nutrifit project is detect 15 food categories.
Details about data for this project available in dataset above.
- beef curry
- chicken nugget
- french fries
- green salad
- grilled salmon
- hamburger
- hot dog
- natto
- omelet
- pizza
- rice
- rice ball
- spaghetti
- steak
- waffle
mosaic image augmentation only implemented on YOLO not on EfficientDet
- all model train and evaluate using FOOD15 v2 dataset (https://drive.google.com/drive/folders/1c_D7QE6YUB5ILFR1aCpREV8EOFFk1BwD?usp=sharing)
map 0.5
as single matrics (bigger better)- subject to:
ukuran*
(free),kategori
(15 food, higher better), danwaktu prediksi**
(5 sec, less better)
name | ukuran (MB) | kategori | waktu prediksi (s) | map 0.5 (val) | map 0.95 (val) | epoch / iterasi | weights files |
---|---|---|---|---|---|---|---|
Efficientdet-d0 | 15 | 79.40 | 0.462 | 8000 | tf (saved) | ||
Efficientdet-d0 food15v3 | 15 | 65.40 | 0.386 | 4500 | tf (saved) | ||
Efficientdet-d1 | 15 | **LACK OF MEMORY** |
|||||
YOLOv4-Tiny | 23.09 | 15 | 74.26 | 5452 | tf (saved) | ||
YOLOv4-fp16 | 122 | 10 | 1000 | tf (saved) | |||
YOLOv4 | 244 | 10 | 67.57 | 1000 | tf (saved) | ||
YOLOv5s-transformer | 14.4 | 15 | 82.7 | 54.8 | 301 | pytorch (.pt) | |
YOLOv5s-transformer-last | 15 | 49.5 | 24.4 | pytorch (.pt) | |||
YOLOv5s6 |
23.98 |
15 |
85.7 |
61.16 |
111 |
pytorch (.pt) |
|
YOLOv5s6-last | 15 | 60.1 | 35.3 | pytorch (.pt) | |||
YOLOv5s-best | 14.8 | 15 | 80.3 | 49.7 | 300 | pytorch (.pt) | |
YOLOv5s-last | 15 | 44 | 22.8 | pytorch (.pt) | |||
YOLOv5m6-best | 15 | 62.2 | 28.8 | pytorch (.pt) | |||
YOLOv5m6-last | 15 | 62.1 | 30.5 | pytorch (.pt) | |||
YOLOv5m6-best-2 | 15 | 39.8 | 24 | pytorch (.pt) | |||
YOLOv5m6-last-2 | 15 | 38.8 | 23.3 | pytorch (.pt) | |||
YOLOv5m-best | 15 | 49.8 | 30.4 | pytorch (.pt) | |||
YOLOv5m-last | 15 | 53.4 | 32 | pytorch (.pt) |
*
this metrics no longer measured/track as project plan has change
**
this metrics measure as a whole user experienced
All weights files in this link https://drive.google.com/drive/folders/1H1M3BRpyGXHtsOhGQx3AHhNlEThmTDh_?usp=sharing
We are select YOLOv5s6
model for our app.
The model reaches highest map 0.5 on validation data and the model meet our criteria: capable to detect 15 food categories,
and based on our integration testing waktu prediksi is less than 5 s.
We are aware the model not implemented in Tensorflow Framework. Base on the capstone project rules we are allowed to not use Tensorflow when it is
not available
. In our case the pre-train model of YOLOv4 and YOLOv5only available
andposible
on darknet and pytorch. We are using pre train model (transfer learning) for faster training, faster experiment, sweet able for GPU limitation on google colab, and equal MAP compare to training from scratch. We arenot possible
to selectYOLOv4-Tiny
(model on Tensorflow format) because the MAP far away lower thanYOLOv5s6
.
The result of the selected model inference https://drive.google.com/drive/folders/1GJOphru8FKrkiW_ihhpe3xnv8A1S8ViS?usp=sharing
We decide to deploy the selected model on Virtual Machine and communicate to users using ResAPI. This API endpoint recive .jpg image sent from the android app backend and returns a list of food detected on the image.
strat flask server $ nohup python resapi.py --port 4040 &
send a request to server $ curl -X POST -F image=@path/to/iamge.jpg http://flask-server.url:P.O.R.T/v1/object-detection/yolov5s6/
just run the notebook .ipynb on goole colab env. and dont forget to add the dataset from (https://drive.google.com/file/d/1lF_9VNyNVDcD8QxQivzOzCn_4LnkZheJ/view?usp=sharing)
- set a VM instance
- install requirements.txt
- download model files from (https://drive.google.com/drive/folders/1H1M3BRpyGXHtsOhGQx3AHhNlEThmTDh_?usp=sharing)
- makesure model path in
resapi.py
pointing the model file - run server
$ python resapi.py
- send a request
- Datasets (allow us to use for non-commercial research purpose) http://foodcam.mobi/dataset256.html
- Yolov5 implementation https://github.com/ultralytics/yolov5
- Yolov4 implementation https://github.com/AlexeyAB/darknet/ and https://github.com/roboflow-ai/darknet
- Shout out to roboflow team for giving away colaborative feature for free