/Handwritten-Equation-Solver

An application to solve handwritten mathematical equations using deep learning algorithms.

Primary LanguageJavaScript

Handwritten Equation Solver Application

ToDo

  • Make a Server for Digit Recognition
  • Make a client App for Digit Recognition

Technologies/Frameworks Used

Server

  • Keras
  • Nodejs
  • Express
  • Keras-js
  • multer
  • pngjs
  • Other common python libraries : numpy, matplotlib etc
  • Other common npm libraries : bodyparser, fs, path etc

Client

ToDo

Prerequisites

Following must be installed

  • Tensorflow
  • Keras
  • Nodejs
    Run npm install in Server/digit_reco_server to install :
    • keras-js
    • pngjs
    • multer
    • Other common nodejs libraries like express etc
  • h5py - To enable storing models on disk
  • matplotlib
  • numpy

Server

The server is built on Nodejs. The Multi-layer perceptron model is built using Keras.

1. Build the model using Keras

cd Server/digit_reco_server/keras_model
python data_gen.py
python encoder.py models_generated/model.hdf5

Doing this will :

  1. Download the MNIST dataset
  2. Construct a simple multi layer perceptron model trained on MNIST dataset
  3. Store the model in models_generated folder in a format that can be used by keras-js for prediction

2. Start the server

cd Server/digit_reco_server
npm start

3. Upload Image

Properties of image to be uploaded :

  • Dimensions : 28 X 28
  • Format : png (with data in RGBA)

Images can be generated from MNIST dataset using Utilities/image_gen.py :

cd Utilities
python image_gen.py

Images are saved in Utilities/Images folder. These images can be uploaded to the server when launched. Any image uploaded must have same properties as those generated by image_gen.py

Upon uploading, the server returns the predicted digit in the image.

Route used : /imgupload

This route should be used by any client application that uses this server.

Request

  • Method : Post
  • Request body : The PNG image
  • URL : /imgupload

Response

The predicted digit

Changing/Updating Keras Model

To change or update the model used for prediction, ideally only the following files must be changed :

  • data_gen.py and model_gen.py in Server/digit_reco_server/keras_model
  • The function interpret_result() in Server/digit_reco_server/predictor/use_keras_model.js

    After making the required changes, the model must be compiled and saved as described above

Client

ToDo