- Sarah Nisar Tisekar - sarahtisekar@gmail.com
- Deep Vora
- Pranav
- Vedant Paranjape
Percy is a console AI based chatbot that uses python, neural networks and deep learning, Tensorflow, keras, openCV, Speechrecognition and Natural language toolkit. The user can communicate with Percy using text, speech or sign language. In addition to features like chatting, telling the time,jokes and funfacts, showing the news and browsing the web, Percy can recognize the user's emotions based on the user's facial expressions or speech.
- GitHub repo link: Link To Repository
- Drive Link(Demo Video): Drive Link here
- Python
- Tensorflow
- openCV
- CNN
- RNN
- Machine Learning
clone the CoC_PeRKS repository
git clone https://github.com/sarah-nisar/CoC_PeRKS
Python, openCV and tensorflow for training InceptionV3 model, a convolutional neural network model for classification.The framework used for the CNN implementation can be found here: Simple transfer learning with an Inception V3 architecture model by xuetsing.The project contains the dataset (1Go).
See requirements.txt and Dockerfile for versions and required APT packages
docker build -t hands-classifier .
docker run -it hands-classifier bash
pip install -r requirements.txt
unzip dataset.zip in the sign-language-hand-gesture-recognition directory To train the model, use the following command:
python train.py \
--bottleneck_dir=logs/bottlenecks \
--how_many_training_steps=2000 \
--model_dir=inception \
--summaries_dir=logs/training_summaries/basic \
--output_graph=logs/trained_graph.pb \
--output_labels=logs/trained_labels.txt \
--image_dir=./dataset
If you're using the provided dataset, it may take up to three hours.
To test classification, use the following command:
python classify.py path/to/image.jpg
This aims to classify the emotion on a person's face into one of seven categories, using deep convolutional neural networks. It is an implementation of this research paper. The model is trained on the FER-2013 dataset which was published on International Conference on Machine Learning (ICML). This dataset consists of 35887 grayscale, 48x48 sized face images with seven emotions - angry, disgusted, fearful, happy, neutral, sad and surprised.
To install the required packages, run
pip install -r requirements.txt
This is currently compatible with tensorflow-2.0 and makes use of the Keras API using the tensorflow.keras library. .
Download the FER-2013 dataset from here and unzip it inside the src folder. Rename the folder to Data.
To train the model,
cd src
python emotions.py --mode train
If you want to view the predictions without training again, you can download the pre-trained model from here
Install the dependencies using the following command:
pip install -r requirements.txt
In addition to the above requirements you might also require ffmpeg if you want to add more sample audio.
Four Datasets(RAVDESS, TESS, EMO-DB, Custom) are already downloaded and already formatted here
All the above mention commands and training have to be executed only once.
Python==3.6
Tensorflow==1.12.0
tflearn==0.3.2
numpy==1.19.0
nltk==3.5
since TFLearn requires Tensorflow (version 1.0+) to be installed and tensorflow.contrib is being removed in version 2.0, two conda environments have been created. One with Tensorflow version 1.12.0 and the other with Tensorflow version 2.2.0.
replace newenv in the trial.sh and trial1.sh to the name of the anaconda environment which has tensorflow version 2.2.0.
To run the Project:
python project2.py
you can then start communicating with the bot.
- Chatbots are being made to ease the pain that the industries are facing today. The purpose of chat bots is to support and scale business teams in their relations with customers. It could live in any major chat applications like Facebook Messenger, Slack, Telegram, Text Messages, etc.
- sign language recognition can be of great value to the disabled.
- A key step in the humanization of robotics is the ability to classify the emotion of the human operator.One possible application for this lies in the area of surveillance and behavioural analysis by law en-forcement.Furthermore such techniques are used in digital cameras to automatically take pictures when the user smiles.
percy which is presently console based can be given a proper UI and can be made into a web app.
percy can be trained to answer more questions by adding more patterns and responses in the intents2.json file, and thus training it better.
One very promising applications involves the humanization of artificial intelligent systems.If Percy is able to keep track of the mental state of the user,it can react upon this and behave appropriately.
Another interesting thing that could be tried is to use Percy to Automate Interviews.