Handwritten Captcha Detection (Emoji and Text)

This is an Emoji and Text Recognizer made for MOSAIC, Udyam'22, organized by IIT(BHU).

Team Name: MoCa

Team members:
Debasish Chakraborty
Vedant Gupta
Nachiketa

Problem Statement:
The task is to automatically detect what is written in the Captcha given to you. Generally simple captchas consist of numbers and letters in a distorted format, but there is a twist to the PS. Instead of numbers, the captcha will consist of letters and emojis!

You have to map the emojis in the captcha to numbers, i.e. when you print the output, you need to print the number corresponding to the emoji detected in the captcha, The mapping will be as follows:
Checkmark : 1
Cloud: 2
Croissant: 3
Heart: 4
Laugh: 5
Smile: 6
Sun: 7

Dataset Used:
For Letters: https://www.kaggle.com/datasets/crawford/emnist
For Emojis: https://drive.google.com/drive/folders/1sw0XVroXFh

We tested our model on various sample images and here are the results


Note :- All the images are drawn on Microsoft Whiteboard


sample_1

sample_2

sample_3