The fact that many young people and children spend a lot of time in front of screens in modern society is nothing new. They spend much of their screen time on social media where they are exposed to explicit content including violence, bullying and exclusion. Our final project aims to create a platform that allows concerned parents to track and monitor their children's behavior and involvement in such acts of violence, by giving us a permission to scan posts and videos of their children Instagram accounts. By using computer vision and machine learning tools to scan for and recognize signs of violence, we believe we can increase parental awareness and reduce incidents like this.
VioREACH.sample.mp4
- We created a full stack web application to enable a parent to register to our system, and by creating a user and granting access to his child Instagram user account, we are able to run Machine Learning and Computer Vision components on each video to identify and categorize different forms of violence.
- The first model we used to identify acts of violence was YOLOv8. We collected a dataset of violent and non-violent videos to train the model. If violence is detected, we defer categorization of the video for further investigation.
- To improve the accuracy of the algorithm in case it detects violent content, we used several classifiers performed on violent videos:
- Pose estimation which is based on the MoveNet algorithm. It's built for speed, so it's good for real-time apps and to our purposes. Within the pose estimation component we implemented mathematical computations (such as punch detection in an interaction between two people) to gather more information about the situation occurring in the video.
- Presence of blood using SVMs (support vector machines).
- Scrum detection using a different YOLOv8 trained model as well.
- A concept called "stacking" is then used to achieve the end result, if any, of categorized violence. The idea is to stack all the small and weak feature outputs and combine them into a final vector to feed into the final classifier. We found the XGBoost classifier to be a perfect match for this task.
- Full integration with the Instagram Basic Display API.
- Creating a MongoDB schema with Mongoose framework.
- Implementation of JWT encryption/decryption on sign-up and login pages.
- Third-party library 'axios' to make HTTP requests from the frontend to the backend.
- Using Material UI components to take advantage of the built-in responsiveness of components to design the front-end website.
- The table below contains all the technologies used for the VioREACH integration:
| Field | Stack |
|---|---|
| Frontend Development |      |
| Backend Development |      |
| Databases |  |
| AI/ML |        |
| IDE's |   |
| Others |   |
- Apply refactoring to the code.
- Improve the accuracy of the blood detection classifier.
- For better results, adding additional classifiers (such as Sound Processor).
- Scan entire profile posts instead of just one.
- Build a sub-system that supports direct notifications to parents' smartphones.
Will be edited further in the future if neccesary
