True story!
A friend of mine (let's call him Carmelo, coz it sounds like watermelon) is ruling a small garden with watermelon plants (digression: I love eating watermelon 😸). However, it wouldn't be a good scenario if everything was oki koki :
- Watering and putting fertilizer is inefficient, because it doesn't target the right places where water and fertilizer are needed, and these resources are expensive
- In early stage, the fruits are vulnerable to insects attacks :
- When the fruits reached maturity, they receive the visit of unexpected watermelon lovers, the crows, see results of their lovely visits :
I thought for a while and came with (the not so special) idea that using machine learning + IoT can help in most of the above problems. For example an object detection computer vision model can help in :
- More effectively target areas that need water/fertilizer
- Detect/count attacked fruits
- Detect presence of attackers and ring the alarm bell
- Count/estimate number of fruit in a given area
- ...
First step in data science project is problem definition. Here I wanted to start simple and build a simple app (web, mobile ? I dunno yet ¯\(ツ)/¯) around the watermelon detection in the wild.
After problem definition, next logical step is to acquire data. To spice things up, I decided to create and label a small dataset. After all:
So, Carmelo (remember him ?) recorded a short video of the plants, of about two and half minutes. After a little investigation about opensource image labelling tools, I found CVAT, a tool by Intel, to be the right one for me in terms of :
- easy to install: through simple a docker-compose service
- intuitive : it offers a simple web interface for labelling and also for labellers accounts administation
- functionalities : the main ones used in computer vision such as segmentation masks, key points, bounding boxes, ...
After about an equivalent of a day of working, I managed ot label around 4.7K images with bounding boxes :
Labelling is an exhausting, task, and I needed to go back to some images multiple times to adjust the boxes.
Data being created, I searched for a good object detection model that offers a good trade-off between accuracy and speed, as I may want to deploy the model on mobile/edge devices later. Yolo V5 is one of the best in this area, so I sticked with it.
Check this Colab notebook to see how to train diffent, Yolo v5 models end-to-end, from data download until model evaluation and conversion.
The trained model is deployed on streamlit and can be accessed through this link
Next steps :
- Label new data to:
- detect attacked watermelon
- create a segmentation map around the attacked area
- count number of healthy/attacked fruits in a given field area
- Find a way to make the models useful for our friend Carmelo, for example by embedding the app on mobile/Raspberry Pi camera, ...