The main objective of this project is to build and deploy a Image Classification ML model for Scones Unlimited, a scone delivery logistics company using Amazon SageMaker.
Build and Monitor a Machine Learning Workflow For Scones Unlimited On Amazon SageMaker.
This project was a part of the project assessment in the 'AWS x Udacity's Machine Learning Engineer Nanodegree Scholarship Program'.
- starter.ipnyb: This Jupyter notebook contains a machine learning working workflow for Image Classification. This includes the necessary preprocessing of the required scones unlimited image datasets, model training, deployment and monitoring using Amazon SageMaker and other associated AWS Services.
- starter.html: Web-page displaying 'starter.ipynb'.
- Lambda.py: assembling the required "lambda.py" scripts for three AWS Lambda functions to build a Step Functions workflow. (Note: The 'lambda_handler' function, normally included in the 'lambda.py' file, serves as the entry point for the Lambda function when it is called by an event like an HTTP request or a scheduled cron job. The 'event' object, which has details about the triggering event, and the 'context' object, which contains details about the current execution environment, are sent to this function. The basic functionality of the Lambda function is carried out in the 'lambda_handler' function, which can also interface with other AWS services, carry out calculations, and process data. The Lambda function may also provide a response to the service or client that called it.)
- Screenshot showcasing working step function.png: screen shot of working step function.
- step function.json: Step Function exported to JSON
Python 3 (Data Science) - v3.7.10 kernel ml.t3.medium instance Python 3.8 runtime for the AWS Lambda Functions
- For local development, you will need to setup a jupyter lab instance.
-
Follow the jupyter install link for best practices to install and start a jupyter lab instance. If you have a python virtual environment already installed you can just pip install it.
pip install jupyterlab
- There are also docker containers containing jupyter lab from Jupyter Docker Stacks.
With the aid of AWS Step Functions and Lambda functions, the project seeks to create a machine learning model for picture categorization by automating a variety of machine learning activities, including data preparation, model training, deployment, and inference.
- The serializeImageData lambda function (zipped lambda script)will copy an object from S3, base64 encode it, and then returns it to the step function as a serialized JSON object in an event.
- The Image-Classification lambda function (zipped lambda script) takes the image output from the previous function, decode it, and then pass inferences back to the the Step Function.
- The Filter-Low-Confidence-Inference lambda function (zipped lambda script) takes the inference data from step 2, and filters only the images that meet the pre-defined threshold.
