/amazon-sagemaker-autopilot-feature-engineering-transformer-and-model-explainability

This repository contains sample code to generate SHAP plots out of the SageMaker autopilot.

Primary LanguageJupyter NotebookMIT No AttributionMIT-0

Amazon SageMaker Autopilot Feature Engineering Transformer and Model Explainability

Amazon SageMaker Autopilot automatically identifies an end-to-end workflow pipeline consisting of data pre-processing, feature engineering, as well as model searching, tuning, and training procedures. The output artifacts can be deployed easily to Amazon SageMaker Hosting Services for operation. Sometimes, however, machine learning tasks may demand more explainability into the pipeline, or just want to leverage a part of the workflow rather than end-to-end.

amazon-sagemaker-autopilot-feature-engineering-explainability.ipynb provides an example on how to leverage the artifacts generated out of the Amazon SageMaker Autopilot, to achieve data transforming, and model explainability separately. In the notebook, it demonstrates how to map the feature name(s) before and after the feature engineering steps of the Amazon SageMaker Autopilot, which is significant for feature importance interpretation and explainability. Also the notebook illustrates how to enable the Amazon SageMaker Debugger, using the best model hyper-parameters identified by the Amazon SageMaker Autopilot, to generate shapely plot visualisation for global and individual prediction explainabilities.

Reference

  1. ML Explainability with Amazon SageMaker Debugger
  2. Amazon SageMaker Autopilot – Automatically Create High-Quality Machine Learning Models With Full Control And Visibility
  3. Amazon SageMaker Solution for explaining credit decisions

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.