Given a picture of several objects on a light horizontal surface and a polygon as input, determine whether it is possible to place all these objects simultaneously on the plane so that they fit into the polygon.
Algorithm input:
- The path to a file with tif, png or jpg extension containing a three-channel RGB image on which on the horizontal
- light surface there are objects from a known set;
- List of pairs of numbers(float) describing the polygon.
- [Optional] Mode of polygon representation (string value). Two are supported: pixels(by default) and relative.
Algorithm output:
- True if items can be placed in a polygon, otherwise false.
- Objects cannot intersect;
- Objects must fully fit into the area specified in the algorithm parameter;
- There must be a distance of at least 50 pixels between objects;
- The smaller of the two visible object dimensions must be at least 50 pixels;
- The larger of the two visible object dimensions must be at least 100 pixels;
- The angle of inclination of the chamber from the surface normal should be within 10 degrees;
- The same object can be present in the photo several times. Objects will be considered independently in solving the placement problem;
- The polygon must contain more than 2 corners;
- The large side of the image must be less than 4500 pixels.
Photos of possible items and background image (for second argument of algorithm): https://drive.google.com/drive/folders/1gk90RoNKPqP_lwWpY8cYFQpLThSHtY2m?usp=sharing
Dataset for testing: images with jpg extension and csv file of input/output data: https://drive.google.com/drive/folders/1XGFnvI2nywZ0e6qcOO4kyAmzYbnEYxoU?usp=sharing
- clone repository
- upload photos of possible items with masks from drive and put it into
intelligent_placer_lib/data/. If you want to use yor own data, you could put it (images and masks) into this dir.
for testing:
- put dataset into
test/data - put dataset specification json into
test/data. Example of such file already put here.
The algorithm for finding objects:
- Get blurred image using
skimage.filters.gaussian(image, 5)and convert it into grayscale - Get the contours of the image using the
skimage.filters.sobel - Remove the noise from the obtained image of the contour by binarization
- Represent contours as a polygon using
cv2.findContours - Consider only contours with area no less than 50*50 pixels, because the minimum width of the object is 50 pixels according to the stipulation
- Match each found contour with the object whose contour is most similar to the given one. If all the contours are not similar, remove the current one from consideration
- Construct a descriptor (a vector of 128 numbers) for each point on the contour of the object to be identified
- Construct a descriptor for each point on the contour of the matched object
- Solve the assignment problem for the distance matrix between these points and map the contour points to each other
- Find the homography matrix and project the mask of the object onto the mask of the size of the input image. Assign pixels of the mask to the label values of the matched object class
- If the homography matrix is not found, or the points of the projected mask are out of bounds, or these points intersect with another segmented object, we consider that the object is not identified, otherwise we add the identified contour to the set of "good" contours
The algorithm for place problem:
- Sum areas of all "good" contours
- Get the area of input polygon
- If the sum less than the area of input polygon, then return true, otherwise - false
- reduce size of items images so theirs contours will be calculated faster
- read all images in memory once
- set maximum number of contour points (because we have two items with a circular shape)
- Consider that pixel in item mask could be addressed with area (set of pixels) in result mask
- replace naive algorithm of placing with the arranging of objects by parallel transfer
- use physics solutions: place objects randomly to polygon and use potential field method
- implemented new method od placing objects: objects are placed recursively with movements and rotations by 5 degree per iteration
- modified classification method: now contours get less impact and descriptors - more, calculated distances of pairs and only "good" pairs are considered
- added exceptional check for red circle and blue phone(rectangle)
- resized input images so algorithm works faster
- started implementation of genetic algo for placing problem
- extended dataset (added more examples with rotations and included example with shadow impact where algorithm don't work)
- add the ability to parallelize evaluations in brute force parallel-rotate solution
- implement genetic algo
- add exceptional check for keychain pass
- evaluate precicion, accuracy and recall automatically (only manual for now)