Easily select icons on the screen in any environment.
This is part of the Hands Free Computing project. Built with OpenCV 3.12 and Python 3.8.
- Clone the repo and open it locally:
$ git clone https://github.com/luiszugasti/IconMatch/
$ cd IconMatch
- Install the requirements:
$ pip install -r requirements.txt
You can use the functions as shown in demo.py as a default entry point.
In the below example, the main set of functions is called within a callback function, as this allows the threshold value to be controlled from a GUI in OpenCV.
import cv2 as cv
from icondetection.demo.demo import render_rectangles, candidate_rectangle_demo
from icondetection.box import grayscale_blur, canny_detection, group_rects
src = cv.imread("source to your image file")
def threshold_callback(val):
"""
Takes a value of threshold for the canny edge detector and finds the
bounding rectangles of appropriate edges within an image.
"""
# accept an input image and convert it to grayscale, and blur it
gray_scale_image = grayscale_blur(src)
# determine the bounding rectangles from canny detection
_, bound_rect = canny_detection(gray_scale_image, min_threshold=val)
# group the rectangles from this step - variable is global for demo purposes
global grouped_rects
grouped_rects = group_rects(bound_rect, 0, src.shape[1])
# (for display purposes) use the provided rectangles to display in your program
render_rectangles(grouped_rects, src.copy(), "Grouped Rectangles", desired_color=(36, 9, 14))
render_rectangles(bound_rect, src.copy(), "Original Rectangles", desired_color=(96, 9, 104))
candidate_rectangle_demo()- Detection of areas with a high likelihood of being clickable icons.
- Detection of closest rectangle to point of interest (be it gaze, or mouse as in the examples)
The current available APIs encompass what your image processing pipeline should contain. Both APIs are currently still experimental as I learn more about OpenCV and optimize code.
Performs canny detection when given a gray scale image and a minimum threshold for hysteresis. Returns bounding rectangles of points of interest.
Groups rectangles that are overlapping in two-dimensional space and returns their conglomerate components.
- Detect regions of interest with moderate accuracy
- Detect candidate region based on proximity
- Detect icon-like objects on the screen
- Context provision into regions of interest
Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are genuinely appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.
Luis Zugasti - @luis__zugasti
Project Link: https://github.com/luiszugasti/IconMatch