In this workshop, you will go through the steps required to build a machine learning application on AWS using Amazon SageMaker.
You will learn how to start experimentation in the SageMaker Studio environment using a familiar JupyterLab notebook experience and run your local code as a SageMaker Training job using the remote function feature. You will also learn how to use SageMaker Studio's Code Editor, which is based on Visual Studio Code – Open Source (Code-OSS), to deploy the model into an endpoint and build an end to end pipeline. You wull also learn how to build an HTTP endpoint using AWS Lambda and Amazon API Gateway to serve inference requests from a web client.
This workshop covers some of the new features announced at AWS re:Invent 2023. To learn more about the new features, watch the session recording.
AWS re:Invent 2023 - Scale complete ML development with Amazon SageMaker Studio (AIM325)
The machine mearning process is an iterative process consisting of several steps:
- Identifying a business problem and the related machine learning problem
- Data ingestion, integration and preparation
- Data visualization and analysis, feature engineering, model training and model evaluation
- Model deployment, model monitoring and debugging
These steps are usually repeated multiple times to better meet business goals after the source data changes or performance of the model drops, for example.
The following diagram shows how the process works:
After you deploy a model, you can integrate it with your own application to provide insights to the end users.
Amazon SageMaker is a fully-managed service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale.
Amazon SageMaker removes the complexity that holds back developer success with each of these steps; indeed, it includes modules that can be used together or independently to build, train, and deploy your machine learning models.
You will use the AI4I 2020 Predictive Maintenance Dataset from the UCI Machine Learning Repository. This synthetic dataset, which contains predictive maintenance data encountered in industry, consists of 10,000 records and 14 features. The features include various measurements collected from machinery and indication of whether the mechine is likely to fail. This basic dataset oversimplifies a predictive maintenance task. However, it keeps this workshop easy to follow while being a good representative of the various steps of the machine learning workflow. You can adapt the steps in this workshop to solve other machine learning tasks, including generative AI fine-tuning and deployment.
In this workshop, your goal is to build a simple machine learning model that predicts whether a piece of machinery is going to fail.
Following is an excerpt from the dataset:
| UDI | Product ID | Type | Air temperature [K] | Process temperature [K] | ... | Machine failure |
|---|---|---|---|---|---|---|
| 1 | M14860 | M | 298.1 | 308.6 | ... | 0 |
| 2 | L47181 | L | 298.2 | 308.7 | ... | 0 |
| 3 | L47182 | L | 298.1 | 308.5 | ... | 0 |
| 51 | L47230 | L | 298.9 | 309.1 | ... | 1 |
The binary (0 or 1) nature of the target variable, Machine failure, suggests you are solving a binary classification problem. In this workshop, you will build a logistic regression model, which will predict a continuous value in the range [0,1]. Using a regression model to solve a binary classification problem is a common approach. The predicted score indicates the system’s certainty that the given observation belongs to the positive class. To make the decision about whether the observation should be classified as positive or negative, as a consumer of this score, you can interpret the score by picking a classification threshold (cut-off) and compare the score against it. Any observations with scores higher than the threshold are then predicted as the positive class and scores lower than the threshold are predicted as the negative class. To learn more about this approach, read https://docs.aws.amazon.com/machine-learning/latest/dg/binary-classification.html.
This diagram shows what you will be building in this workshop:
This workshops consists of six modules:
- Module 0: Access the AWS Console and clone the GitHub repository.
- Module 1: Use a JupyterLab space in SageMaker Studio to perform experimentation and feature engineering, and build and train a logistic regression model using XGBoost. The model will predict whether the machinery is going to fail.
- Module 2: Use Code Editor in SageMaker Studio to deploy the model to an inference endpoint.
- Module 3: Still using Code Editor, build an end-to-end pipeline to download the data source, perform feature engineering, train a model, register it in the model registry, and deploy it into an inference endpoint.
- Module 4: Build an HTTP API using Amazon API Gateway and an AWS Lambda function to invoke the Amazon SageMaker endpoint for inference.
- Module 5: Use a web client to invoke the HTTP API and perform inference.
Please follow the order of modules because the modules depend on the results from the previous modules.
If you are attending the Scale complete ML development with Amazon SageMaker Studio workshop run by AWS, the AWS event facilitator has provided you access to a temporary AWS account preconfigured for this workshop. Proceed to Module 0: Open SageMaker Studio.
If you want to use your own AWS account, you'll have to execute some preliminary configuration steps as described in the Setup Guide.
⚠️ Running this workshop in your AWS account will incur costs. You will need to delete the resources you create to avoid incurring further costs after you have completed the workshop. See the clean up steps.
Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
Giuseppe A. Porcelli - Principal ML Specialist Solutions Architect - Amazon Web Services
Antonio Duma - Senior Startup Solutions Architect - Amazon Web Services
Hasan Poonawala - Senior ML Specialist Solutions Architect - Amazon Web Services
Mehran Nikoo - Senior Solutions Architect - Amazon Web Services
Bruno Pistone - AI/ML Specialist Solutions Architect - Amazon Web Services
Durga Sury - ML Solutions Architect - Amazon Web Services