/Synthetic-Data-Generator-for-Retail-Products-Detection

A synthetic data generator for retail object detection, which is built with Blender and Python.

Primary LanguagePythonMIT LicenseMIT

Synthetic Data Generator for Retail Products Detection

Training convolutional neural network models requires a substantial amount of labeled training data to achieve good performance. However, the collection and annotation of training data is a costly, time-consuming, and error-prone process. One promising approach to overcome this limitation is to use Computer Generated Imagery (CGI) technology to create a virtual environment for generating synthetic data and conducting automatic labeling.

"Synthetic Data Generator for Retail Products Detection" is an open-source project aimed at constructing a synthetic image data generation pipeline using Blender and Python. The generated synthetic data is utilized for training YOLOv5 models and applied to the task of recognizing retail products. This project leverages Blender to produce randomized synthetic images containing 63 types of retail products (e.g., cereal boxes, soda cans, etc.), and exports corresponding data labels and annotations (2D detection boxes in YOLO format).

Synthetic-Data-Generator-for-Retail-Products-Detection

Installation (Windows 10)

Step1: Install Blender 3.3 LST

Visit the Blender 3.3 LTS web page and click on the Windows – Installer link to initiate the download. Once you have downloaded, install Blender in your PC.

Step2: Download this repo

Download this repo via git

git clone https://github.com/MichaelLiLee/Synthetic-Data-Generator-for-Retail-Products-Detection.git

or download via ZIP file.

Step3: Download assets

Before using the synthetic data generator to generate data, it is necessary to prepare digital assets for creating virtual scenes.These digital assets include retail product models, background and occluder 3d models, PBR materials, and lighting.

In this project, the required digital assets to be prepared are as follows:

  1. 3D models of 63 retail products(.blend file), You can download from this Google Drive Link.
  2. 3D Models with No Texture Serving as background and occluder (.blend file), You can download from this Google Drive Link.
  3. 10 PBR materials from ambientCG, You can download from this Google Drive Link. You can also download more materials manually from the AmbientCG website. Alternatively, you can use this python script from BlenderProc to download them.
  4. 10 HDRIs from PolyHaven, You can download from this Google Drive Link. You can also download more HDRIs manually from the PolyHaven website. Alternatively, you can use this python script from BlenderProc to download them.

Once you have downloaded these digital assets, please place the assets in the corresponding folders in the following order:

retail products(.blend) >> Assets/foreground_object

background and occluder(.blend) >> Assets/background_occluder_object

pbr_texture(folders contain a series of jpg images) >> Assets/pbr_texture

HDRIs(.exr) >> Assets/hdri_lighting

Step4: Setting paths

After completing the aforementioned steps, it is necessary to set several parameters related to the path in theSDG_200_SDGParameter.pyfile:

  1. blender_exe_path : The path to the blender executable(default: C:/program Files/Blender Foundation/Blender 3.3/blender).
  2. asset_background_object_folder_path : The path to background object assets(default: Assets/background_occluder_object).
  3. asset_foreground_object_folder_path : The path to foreground object assets(default: Assets/foreground_object).
  4. asset_ambientCGMaterial_folder_path : The path to the downloaded ambientCG PBR materials(default: Assets/pbr_texture).
  5. asset_hdri_lighting_folder_path : The path to the downloaded Poly Haven HDRIs(default: Assets/pbr_texture).
  6. asset_occluder_folder_path : The path to occlusion object assets(default: Assets/background_occluder_object).
  7. output_img_path : The path where rendered images will be saved(default: gen_data/images).
  8. output_label_path : The path where YOLO format bounding box annotations will be saved(default: gen_data/labels).

Step5: Test installation was successful - Run SDG to generate synthetic data

After completing the paths settings, execute the SDG_400_Looper.py file in vscode.

When rendering images and generating data labels, image and label files will be generated in thegen_datafolder.

Usage

1.Setting parameters in SDG_200_SDGParameter.py

This python file contains a configuration class to configure this blender-based synthetic data generator pipeline, The following parameters can be adapted to your specific application.

Category Parameter Description Distribution
Dataset sizegen_numThe quantity of synthetic images needed to be generated.Constant(10)
3D Objectasset_background_object_folder_pathA set of 3D models containing various simple geometric shapes such as cubes and cylinders.A set of 3d model assets(10)
background_poisson_disk_sampling_radiusBackground objects separation distance.Constant(0.2)
bg_obj_scale_ratio_rangeThe distribution of the scale ratio of background objects within the blender scene.Constant(2.5)
asset_occluder_folder_pathA set of 3D models containing various simple geometric shapes such as cubes and cylinders.A set of 3d model assets(10)
occluder_areaSpatial distribution area of occlusion objects.Cartesian[Uniform(-0.6, 0.6), Uniform(-0.4, 0.4), Uniform(1.5, 1.9)]
occluder_poisson_disk_sampling_radiusOcclusion objects separation distance.Constant(0.25)
num_occluder_in_scene_rangeThe distribution of the number of occlusion objects within the blender scene.Uniform(5, 10)
occluder_scale_ratio_rangeThe distribution of the scale ratio of occluder objects within the blender scene.Uniform(0.5, 2)
asset_foreground_object_folder_pathA set of 63 retail items 3D assets.A set of 3d model assets(63)
foreground_areaSpatial distribution area of foreground objects.Cartesian[Uniform(-1.25, 1.25), Uniform(-0.75, 0.75), Uniform(0.5, 1)]
foreground_poisson_disk_sampling_radiusForeground objects separation distance.Constant(0.3)
num_foreground_object_in_scene_rangeThe distribution of the number of retail items within the blender scene.Uniform(8, 20)
fg_obj_scale_ratio_rangeThe distribution of the scale ratio of foreground objects within the blender scene.Uniform(0.5, 2.2)
-Random rotation angle to background and occluder objects.Euler[Uniform(0, 360), Uniform(0, 360), Uniform(0, 360)]
-Random unified rotation angle to all foreground (retail products) objects.Euler[Uniform(0, 360), Uniform(0, 360), Uniform(0, 360)]
Textureasset_ambientCGMaterial_folder_pathA set of PBR materials that are randomly applied to the surfaces of the background and occluder objectsA set of PBR texture assets(10)
Environment Lightingasset_hdri_lighting_folder_pathA set of high dynamic range images (HDRI) for scene lighting.A set of hdri assets(10)
hdri_lighting_strength_rangeThe distribution of the strength factor for the intensity of the HDRI scene light.Uniform(0.1, 2.2)
-Randomly rotate HDRI map.Euler[Uniform(-30, 120), Uniform(-30, 30), Uniform(0, 360)]
Camera & Post-processing-Perspective Camera focal length value in millimeters.Constant(35)
img_resolution_xNumber of horizontal pixels in the rendered image.Constant(1728)
img_resolution_yNumber of vertical pixels in the rendered image.Constant(1152)
max_samplesNumber of samples to render for each pixel.Constant(128)
chromatic_aberration_value_rangeThe distribution of the value of Lens Distortion nodes input-Dispersion, which simulates chromatic aberration.Uniform(0.1, 1)
blur_value_rangeThe distribution of the value of Blur nodes input-Size, which controls the blur radius values.Uniform(2, 4)
motion_blur_value_rangeThe distribution of the value of Vector Blur nodes input-Speed, which controls the direction of motion.Uniform(2, 7)
exposure_value_rangeThe distribution of the value of Exposure nodes input-Exposure, which controls the scalar factor to adjust the exposure.Uniform(-0.5, 2)
noise_value_rangeThe distribution of the value of brightness of the noise texture.Uniform(1.6, 1.8)
white_balance_value_rangeThe distribution of the value of WhiteBalanceNode input-ColorTemperature, which adjust the color temperature.Uniform(3500, 9500)
brightness_value_rangeThe distribution of the value of Bright/Contrast nodes input-Bright, which adjust the brightness.Uniform(-1, 1)
contrast_value_rangeThe distribution of the value of Bright/Contrast nodes input-Contrast, which adjust the contrast.Uniform(-1, 5)
hue_value_rangeThe distribution of the value of Hue Saturation Value nodes input-Hue, which adjust the hue.Uniform(0.45, 0.55)
saturation_value_rangeThe distribution of the value of Hue Saturation Value nodes input-Saturation, which adjust the saturation.Uniform(0.75, 1.25)
chromatic_aberration_probabilityProbability of chromatic aberration effect being enabled.P(enabled) = 0.1, P(disabled) = 0.9
blur_probabilityProbability of blur effect being enabled.P(enabled) = 0.1, P(disabled) = 0.9
motion_blur_probabilityProbability of motion blur effect being enabled.P(enabled) = 0.1, P(disabled) = 0.9
exposure_probabilityProbability of exposure adjustment being enabled.P(enabled) = 0.15, P(disabled) = 0.85
noise_probabilityProbability of noise effect being enabled.P(enabled) = 0.1, P(disabled) = 0.9
white_balance_probabilityProbability of white balance adjustment being enabled.P(enabled) = 0.15, P(disabled) = 0.85
brightness_probabilityProbability of brightness adjustment being enabled.P(enabled) = 0.15, P(disabled) = 0.85
contrast_probabilityProbability of contrast adjustment being enabled.P(enabled) = 0.15, P(disabled) = 0.85
hue_probabilityProbability of hue adjustment being enabled.P(enabled) = 0.15, P(disabled) = 0.85
saturation_probabilityProbability of saturation adjustment being enabled.P(enabled) = 0.15, P(disabled) = 0.85

2.Initiate the synthetic data generation loop via SDG_400_Looper.py

Once the parameter settings are configured, execute the SDG_400_Looper.py file to initiate the synthetic data generation loop.

Additional resource

Real Retail Product Image Dataset for validation purpose : Consisting of 1267 images of real retail products, this dataset originated from the UnityGroceries-Real Dataset. This project has corrected annotation errors and converted the data labels into YOLO format. The dataset can be downloaded from this Google Drive link."

Inspiration

This project is inspired by the Unity SynthDet project, with improvements in methodology (including the addition of PBR materials, HDRI lighting, and ray tracing rendering), and it has been recreated using Blender.

Borkman, Steve, et al. (2021). Unity perception: Generate synthetic data for computer vision.