Sign Language is a non-verbal and visual language that is widely used all over the world by the deaf community for communication with others. In India, the Indian Sign Language (ISL) is widely used by the Indian deaf community to communicate with others. However, problems arise when the hearing-impaired persons attempt to communicate with normal people who use spoken language and have no knowledge of Indian Sign Language. In this paper, a web app, 'Sign Kit' is developed to serve as a complete toolkit for Indian Sign Language by providing different modules to help people work with ISL. The system helps to improve the communication of language-speaking people with deaf people by converting speech and text into animated ISL gestures, and displaying the signs using a 3D avatar. The system also provides facilities for people to learn ISL and to create simple videos using ISL so that the videos can be viewed and understood by the deaf community. The web app is based on the MERN stack, and care has been taken to make sure that the system is excellent from the human-computer interaction perspective. The system received a satisfaction score of 4.44 out of 5, Net Promoter Score of +36, SUS score of 81.5, appearance rating of 4.52 out of 5 and overall rating of 4.32 out of 5. The system also has a high task success rate, an accurate speech recognition module with a WER of only 6.39% and very accurate 3D animations which received a similarity score of 4.87 out of 5 during the survey.
- The repo contains two branches 'main' and 'api'.
- The 'main' branch contains the source code for the React based client app (hosted on Vercel).
- The 'api' branch contains the source code for the NodeJs based API of the system (hosted on Heroku).
- To run the client locally:
- Clone the repo.
git clone https://github.com/spectre900/Sign-Kit-An-Avatar-based-ISL-Toolkit.git
- Checkout into the 'main' branch.
git checkout main
- Enter the 'client' folder.
cd client/
- Install all necessary dependencies.
npm install
- Run the React app locally.
npm start
- The web app will open in your default browser.
- Clone the repo.
- To access the webapp directly:
The project 'Sign Kit: An Avatar based Indian Sign Language Toolkit' has been created as a mini project for the course IT351- Human Computer Interaction.
- Pratham Nayak (191IT241)
- Aprameya Dash (191IT209)