The virtual board project is aimed towards the educational domain to make it easier for instructors to express concepts by drawing and writing with relative ease using hand gestures.
We need the virtual board to make it easy for people to address their explanations to their audience through handwriting and drawing in a real-time environment while also adding other functions to make the board more interactive to the user, the benefit of the virtual board is to remove uneasiness of mouse writing and drawing and also to make it more clear for the audience to follow up with your ideas.
- Pytoch-GPU==1.10.0
- OpenCV==4.5.4
We had to generate our dataset which is around 3500 images using a python script to generate it. But we had to label the dataset manually which was time-consuming so we started using our model to try to predict the bounding boxes points for our new data images and print the annotation as text but we had to check the sanity of the prediction manually to ignore bad data. We also added more images from other datasets found that served our approach and of course, we did tons of augmentation.
Note: This annotation is fitted for only YOLOv5 with PyTorch if you needed to use it with any other model you have to update it using Roboflow.
We used Yolov5 to train an object detection model that recognizes human fingertips to use this as a main milestone in our project.
1-Fit was better than other pre-trained models like (VGG16, ResNet) with real-time testing. 2-Can be used with OpenCV in high performance as it manages users to apply cuda easily for GPU processing with our code.
To run in real-time simply clone the repository and download the weights file and then run the real-time.ipynb file.
Here is the output of the model.
If you have any queries or concerns, please feel free to contact us. Ahmed Ayman, Muhammed Swelam, Ahmed Samy and Ali Hegazy