/flask-ml-azure-serverless

Deploy Flask Machine Learning Application on Azure App Services

Primary LanguagePython

flask-ml-azure-serverless

Python application test with Github Actions

Deploy Flask Machine Learning Application on Azure App Services

Overview of the Project

Functionality and API of Machine Learning Application

The application is implemented in Python by using the Flask framework. It uses a pre-built machine learning model to predict housing prices based on the Bosten Housing Dataset. The prediction functionality is made available as API endpoint "/predict". By sending a feature set formatted as JSON in an HTTP POST request, the predicted price is returned formatted as JSON. Example for querying the API endpoint:

$ curl -d '{
   "CHAS":{
      "0":0
   },
   "RM":{
      "0":6.575
   },
   "TAX":{
      "0":296.0
   },
   "PTRATIO":{
      "0":15.3
   },
   "B":{
      "0":396.9
   },
   "LSTAT":{
      "0":4.98
   }
}'\
     -H "Content-Type: application/json" \
     -X POST "https://flask-ml-service-agaupmann.azurewebsites.net/predict"

{"prediction":[20.35373177134412]}

Note: the following bash scripts can be used to send these requests for yourself.

  • make_predict.sh to query an app running on your local machine
  • make_predict_azure_app.sh to query an app running remotly on another machine

Architecture of Machine Learning Application

Architectural Diagram of the Machine Learning application
Architectural Diagram of the Machine Learning application

The architecture of the Machine Learning Application comprises the following elements and services:

  • GitHub
    • Version Control: Git repository for application's source code and configuration files
    • CI/CD Pipeline: GitHub actions workflow for automated Continuous Integration and Continuous Delivery of the application
  • Azure (Microsoft's Public Cloud)
    • App Service: managed platform for deploying and running web applications.
    • App Service Plan: defines a set of compute resources for a web app to run.
      • Region (West US, East US, etc.)
      • Number of VM instances
      • Size of VM instances (Small, Medium, Large)
      • Pricing tier (Free, Shared, Basic, Standard, Premium, PremiumV2, PremiumV3, Isolated)
    • Resource Group: logical container that holds related Azure resources created for the application.
    • Azure DNS: an Internet URL is assigned to the application for the clients to access the application over the Internet.
    • Cloud Shell: a command line interface (bash or PowerShell) that is available after login into the Azure Portal. It has persistent storage for files and provides a Linux environment that can be used for application development and Azure administration.
    • Azure Active Directory: identity service providing authentication and authorization of users and services.

Project Management

Trello Board

Link to Trello Board: Task Board Udacity

Project Plan

Link to project plan on Google Sheets: Udacity Project Plan

Demo of Application with Screencast

Link to YouTube video: Demo CI/CD Pipeline for Web Application

Description How To Use Project's CI/CD Pipline with GitHub and Azure

Application Development and Testing in Azure Cloud Shell

Azure Cloud Shell Setup

After setting up an Azure account, log into the Azure portal and start the Azure Cloud Shell. Clone the repository into Azure Cloud Shell via HTTPS or SSH. The following screenshot shows the command for cloning via SSH.

Project cloned into Azure Cloud Shell
Project cloned into Azure Cloud Shell

Prepare Environment in Azure Cloud Shell

After having successfully cloned the repository, set up a Python virtual environment with required dependencies (modules) by running following commands.

$ make setup                          # Python virtual environment is created in a hidden subfolder in user's home directory
$ source ~/.azure-devops/bin/activate # Python virtual environment is activated
(.azure-devops)$ make all             # Required Python modules listed in "requirements.txt" are installed, Python files are linted and the ML app is tested
Passing tests after make all
Passing tests that are displayed after running the make all command from the Makefile.

Testing the Application

After making changes to the code or configuration files and before committing and pushing changes to the repository, run unit tests and load tests to make sure that the application is working as expected and without errors.

Unit Tests

Unit tests are configured with pytest and can be run explicitly by executing the command make test. These tests are defined in files tests/conftest.py and tests/unit/test_app.py. Unit tests check the availability and responses of the Flask routes or URLs "/" and "/predict".

Output of a test run
Output of a test run started with make test
Load Tests

Load tests are configured and executed with locust and can be run explicitly by executing one of these commands:

Load testing ML app running on localhost, Page 1
Load testing ML app running on localhost with locust. Testing can also be started with command make loadlocalhost, Page 1
Load testing ML app running on localhost, Page 2
Load testing ML app running on localhost with locust. Testing can also be started with command make loadlocalhost, Page 2

As soon as the test runs in Azure Cloud Shell have been successfully passed, the changes in the application's code files can be pushed to the remote repository on GitHub.

(.azure-devops)$ git commit -am "Description of changes"
(.azure-devops)$ git push

GitHub Actions for Automated CI and CD

The GitHub Actions workflow for Continuous Integration and Continuous Delivery (CI/CD) is started automatically as soon as changes are committed to the GitHub repository. The CI/CD workflow is defined in the YAML file .github/workflows/main.yml in this repository and contains these stages:

  • CI: Set up Python environment
  • CI: Install dependencies
  • CI: Lint Python source code with pylint
  • CI: Run tests with pytest and locust
  • CD: Login to Azure
  • CD: Configure Azure App Services
  • CD: Deploy app to Azure App Services
  • CD: Logout from Azure
  • CD: Run load tests against deployed app with locust
GitHub Actions YAML file for CI and CD, Page 1
GitHub Actions YAML file for CI and CD, Page 1
GitHub Actions YAML file for CI and CD, Page 2
GitHub Actions YAML file for CI and CD, Page 2

Continuous Delivery

Consulting Udacity Support about Using GitHub Actions instead of Azure Pipelines

Note: Microsoft made a change in Azure Pipelines Grant for Public Projects. It set the maximum number of build requests to 0. This means that nothing is built until you pay for it or you can get a free grant. Please see this blog post. According to Xiaodi from the Udacity Student Services Team it’s ok to use Github Actions instead of Azure Pipeline in the project.

Udacity Support Ticket #1385528
Screenshots of Udacity Support Ticket #1385528
Screenshot of Ticket #1385528, Page 2
Screenshot of Ticket #1385528, Page 2 - confirmation by Xiaodi (Udacity Support) is highlighted in blue
Screenshot of Ticket #1385528, Page 1
Screenshot of Ticket #1385528, Page 1
Screenshot of all opened tickets
Screenshot of all opened tickets

Deploy the application using GitHub Actions into Azure App Services

Screenshot of Azure Azure App Service showing deployed ML application
Screenshot of Azure Azure App Service showing deployed ML application
Screenshot of a successful prediction in Azure Cloud Shell
Screenshot of a successful prediction in Azure Cloud Shell

Use the Azure CLI to deploy and manage an application

The Azure CLI can be used for manual deployment of the application into Azure App Services. The following commands can be used to manage the application on the command line.

# Make a dryrun to see if deployment to Azure App Services would be successful
az webapp up -n flask-ml-service-agaupmann --location eastus --sku F1 --dryrun

# Deploy the application to Azure Web Services
az webapp up -n flask-ml-service-agaupmann --location eastus --sku F1

# List all deployed applications
az webapp list

# Redeploy of application, e.g. after implementing new functionality
az webapp up

# Get the details of a web app's logging configuration
az webapp log show

# Start live log tracing for a web app
az webapp log tail

# Remove the application's deployment from Azure App Services
az webapp delete --name flask-ml-service

These Azure CLI commands are also provided in the file commands.sh in this repository.

Test the application inside of GitHub Actions

Test application inside of GitHub Actions workflow
Test application inside of GitHub Actions workflow: the testing steps are "Test with pytest" before deployment and "Run load tests" after deployment
Test application inside of GitHub Actions workflow with pytest
Running unit tests inside of Github Actions with pytest in the workflow step "Test with pytest"

Load test the application using Locust

Test application inside of GitHub Actions workflow with locust, Page 1
Running load tests inside of Github Actions with locust in the workflow step "Run load tests", Page 1
Test application inside of GitHub Actions workflow with locust, Page 2
Running load tests inside of Github Actions with locust in the workflow step "Run load tests", Page 2

Possible Improvements of the Application in the Future

  • Containerization: create a Docker image and publish it on DockerHub
  • Run application on Kubernetes and pull container image from DockerHub
  • Add authentication so that only known users are allowed to access the API
  • A selection of multiple machine learning models could be provided for the user to select when calling the API endpoint