Facial-Emotion-Recognition-using-NodeRED

This repository aims to introduce NodeRED and provide the reader the starting points through which he/she can build a dashboard that utilizes a Facial Emotion Recognition (FER) api and NodeRED as well as generate a website displaying Cards of People of Determination. This was inspired from the application "LOOK AT ME" that is aimed at Autistic children. LOOK AT ME app gamifies interactions, helping users learn to read moods, remember faces, and express themselves with facial expressions and poses. Children with autism spectrum conditions (ASC) have major difficulties in recognizing and responding to emotional and mental states in others' facial expressions. Such difficulties in empathy underlie their social-communication difficulties that form a core of the diagnosis (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2781897/).

Node-RED on IBM Cloud

Node-RED is an open source visual flow-based programming tool used for wiring not only Internet of Things (IoT) components, but also integrating an ensemble of service APIs, including ones provided by IBM Cloud. A node in NodeRED performs a particular functionality, aiming at providing a way to minimize the amount of coding needed to build a given application.

More information about Node-RED, including the documentation, can be found here.

Quick Overview

There are two parts to this repository:

You will be able to use an external package in NodeRED that retrieves emotion based on the face detected within an image and display the dominant facial emotional expression on a dashboard.
You will be able to send an HTTP request to get dataset related to Cards of People of Determination in UAE as well as generate an HTTP end-point to create a simple sample website.

Pre-requisite

An IBM Cloud account - can be created through going to IBM Cloud.
An account on Face++ Cognitive Services to retrieve an API key.

Face++ Cognitive Services API

After creating an account, go to API Key page.
Click + Get API Key.
Enter the required details and click Submit.
Copy generate API Key and API Secret, which you will use within the upcoming flow in NodeRED.

Creating the Node-RED Application

Go to Catalog and select Node-RED Starter under the category Starter Kits

Enter an App name and a unique Host name along with selecting the approprate region/location, organization and space.
Click Create

An instance of the Node-RED application (which uses Node.js to create server-side JavaScript applications) is created to which a Cloudant NoSQL database is bound. It will take some time for the application status to change to Running, indicating that it is running.

Making face-plusplus-node (an external package) to the Function Nodes in NodeRED

Under Overview, click on Enable under Continuous delivery to, as the naming suggests, enable continous delivery
Confirm all the details and click on Create
Click on Eclipse Orion Web IDE to open the online editor

Go to bluemix-setting.js
Find the definition of the functionGlobalContext object and add face-plusplus-node

functionGlobalContext: {
  fpp:require("face-plusplus-node")
},

Define it as a dependency in the package.json file

"dependencies": {
   ...,
  "face-plusplus-node": "latest"
},

Save the modifications, go to Git in the editor, commit and push all changes
Wait for restaging to take place and check if the application is running again

Creating CloudantNoSQL database where data is store

As the database is already created and bound to this application, simply got to Connections and select the database.

Click the name on the right of Alias of to access the Cloudant SQL DB service.

Click LAUNCH CLOUDANT DASHBOARD.
Click Create Database and enter the name of the database (in our case, enter faceplusplus).
Click Create.

Setting Up the Node-RED Application & a Quick Tour

Go back to the application from the IBM Cloud Dashboard and click Visit App URL to access the Node-RED editor used to build the application.
Follow the directions to access the Node-RED editor (you are encouraged to secure your Node-RED editor to ensure that only authorized users can access it).

Click on Go to your Node-RED flow editor. This should open a fresh new flow in the Node-RED flow editor called Flow 1

Creating the flows

Part 1: Facial Emotional Recognition

Go to the hamburger menu and select Manage palette (this is to install nodes needed)
Click on Install tab, look for node-red-contrib-browser-utils, node-red-node-base64, and node-red-dashboard and install each.
Go to the hamburger menu and select Clipboard under Import.
Paste the content emotionrecogition_flow.json from this repository and click Import.

Double click on the function node called Emotion Recognition Face++ and replace and with the Api Key and API Secret that you copied earlier.
Double click on template node called Image and click on the button with pencil icon to add this to a new ui_group.
Confirm the configuration of the new dashboard group config node (make sure the width is 7) and click Add.
Click Done
Double click on template node called Dominant Facial Emotional Expression and added to the Group you created.

You can confirm that the dashboard components were added correctly by view the dashboard tab on the right.

Click Deploy and test it out by click button on the camera node and go to https://.eu-gb.mybluemix.net/ui/ , replacing with your application's hostname, to view the results on the dashboard (you can also click the button with the arrow comming out of the box icon under the dashboard tab to got to the dashboard).

Part 2: Creating a sample website

Paste the content samplewebpage_flow.json from this repository and click Import.

Explore each node to understand what is happening in relation to creating an HTTP endpoint and using an HTTP request node.
Go to https://.eu-gb.mybluemix.net/samplewebapp to check out the webpage you just created.