Bangkit 2023 Company Capstone: How Dicoding Verifies User on Registration and Exams

There are some individuals that exploit the Dicoding system. So far, Dicoding only does email verification and we want to ensure each and every user, including what they do is legitimate. We created a face and gesture recognition system based on webcam-captured video images that covers
1. Face verification and comparison on registration, starting exam, and project submission
2. Gesture recognition and tracking on exam.

Team Members

ID	Name
C002BSY3390	Jundan Haris
C002BSX3335	Faiza Kamilah Setiawan
M002BSY0514	Agung Kurniawan
M002BSX1474	Kezia Nathania Novaleni

Machine Learning

Datasets:

Face Recognition : chaitanyakasaraneni/recognisingfacesinthewild: Kaggle competition by Northeastern SMILE Lab - Recognizing Faces in the Wild (github.com) - https://github.com/chaitanyakasaraneni/recognisingfacesinthewild Gesture Recognition : Synthetic Gaze and Face Segmentation (kaggle.com) - https://www.kaggle.com/datasets/allexmendes/synthetic-gaze-and-face-segmentation

References:

https://medium.com/@rinkinag24/a-comprehensive-guide-to-siamese-neural-networks-3358658c0513

Rane, M. et al. (2023). Real-Time Automated Face Recognition System for Online Exam Examinee Verification. In: Laouar, M.R., Balas, V.E., Lejdel, B., Eom, S., Boudia, M.A. (eds) 12th International Conference on Information Systems and Advanced Technologies “ICISAT 2022”. ICISAT 2022. Lecture Notes in Networks and Systems, vol 624. Springer, Cham. https://doi.org/10.1007/978-3-031-25344-7_34

Singh, A. and Das, S. (2022) ‘A cheating detection system in online examinations based on the analysis of eye-gaze and head-pose’, Proceedings of The International Conference on Emerging Trends in Artificial Intelligence and Smart Systems, THEETAS 2022, 16-17 April 2022, Jabalpur, India [Preprint]. doi:10.4108/eai.16-4-2022.2318165.

Tech Stack Used

Machine Learning

for machine learning task we are using:

TensorFlow
OpenCV
MTCNN for face detections
keras

credit to : https://github.com/yinguobing/head-pose-estimation

Head_pose_estimation models

Frontend

For the frontend part, our tech stack includes:

Vite
React
React router
Chakra UI
Formik
React webcam
html2pdf

Backend

The backend of our system is implemented using Flask, Firebase for authentication and storage, and Google Cloud Storage for file uploads.

Deployment

Both frontend and backend, we use Google Cloud App Engine for deployment.

Getting Started

To set up and run the project locally, clone the repository first using command
git clone https://github.com/jundanha/verifies-user-on-registration-and-exams.git

Run the frontend

Navigate to frontend directory: cd client
Install dependencies: npm install
Run development server: npm run dev

Run the backend

Ensure you have python installed.
Navigate to backend directory: cd backend
Install required packages: pip install install -r requirements.txt
Run the Flask server: python App.py

Remember, to run the backend, you need credentials.json

Thank You.

With Love, C23-VU01 Team