Matching givewn product in the shelf
The problem statement here is to detect product images from a shelf of goods in a super market. The shelf may contain 100 product categories. So, we are asked to execute the detection or product matching task in two ways
- case 1: If aproduct image is given, we need to locate the products inside the shelfimage
- case 2: If shelfimage is given to the system, the system should detect list of known products present in the shelf
More detailed and elaborated problem statement an be found here - Product Detection from Packshots
The dataset contains 3153 shelf images, and 300 product images corresponding to 100 different categories containing 3 images for each of the individual caregory.
- Dataset source - Shelf-Product_dataset
- For Object Detection we have used Yolov4-P5 trained on grocery-shelf images data previously, where the image size is 896
- The next part is product matching, where we trained a classification model using transfer learning on Resnet50V2 trained on 'imagenet', where we added one softmax layer
- For fine-tuning the model and due to very less training data, we freezed the resnet50v2, and trained only softmax conatining 209,400 trainable parameters only.
- To handle lack of training data used data augmentations. For data augmentation we use image-augmentor module, where we utilized the following functionalities-
- Auto-contrast
- Auto-sharpness
- Auto-rotate(-10 to +10 deg)
- Auto-enhance color
- Auto equalize hist
1st apply all on ooriginal image, then applied all on rotated image, so we obtain 10 augmented samples, total samples after augmentation from 300 to 3600.
- The train, validation split used is (valid+test) ratio 0.1665, so train contains 3400, i.e 34 images for each class, and valid and test are split with 50% split ratio. Thus, test and valid contain single image.
- Traained the model for 10 epochs, with LR= 1e-04, batch_size = 16, and got good accuracy on test set.
- After the classifier is trained, we fetched the detection result for shelf images.
- After detection is done, traversed through each bounding box, cropped those and pass to classififcation model, the classifier predicts the product id out of it.