/cloudml-samples

Cloud ML Engine is now a part of AI Platform

Primary LanguagePythonApache License 2.0Apache-2.0

AI Platform Training and Prediction

License

Welcome to the AI Platform Training and Prediction sample code repository. This repository contains samples for how to use AI Platform for model training and serving.

Note: If you’re looking for our guides on how to do Machine Learning on Google Cloud Platform (GCP) using other services, please checkout our other repository: ML on GCP, which has guides on how to bring your code from various ML frameworks to Google Cloud Platform using things like Google Compute Engine or Kubernetes.

Overview

The repository is organized by tasks:

Each task can be broken down to general usage (CPU/GPU) to specific features:

Scroll down to see what we have available, each task may provide a notebook or code solution. Where the code solution will have a README guide and the notebook solution is a full walkthrough. Our code guides are designed to provide you with the code and instructions on how to run the code, but leave you to do the digging, where our notebook tutorials try to walk you through the whole process by having the code available in the notebook throughout the guide.

If you don’t see something for the task you’re trying to complete, please head down to our section What do you want to see?

Setup

For installation instructions and overview, please see the documentation. Please refer to README.md in each sample directory for more specific instructions.

Getting Started

If this is your first time using AI Platform, we suggest you take a look at the Introduction to AI Platform docs to get started.

AI Platform Training

Notebook Tutorial:

  • scikit-learn: Random Forest Classifier - How to train a Random Forest Classifier in scikit-learn using a text based dataset, Census, to predict a person’s income level.
  • XGBoost - How to train an XGBoost model using a text based dataset, Census, to predict a person’s income level.

Code Guide:

Cloud TPUs

Tensor Processing Units (TPUs) are Google’s custom-developed ASICs used to accelerate machine-learning workloads. You can run your training jobs on AI Platform, using Cloud TPU.

Hyperparameter Tuning (HP Tuning)

Notebook Tutorial:

  • scikit-learn: Lasso Regressor - How to train a Lasso Regressor in scikit-learn using a text based dataset, auto mpg, to predict a car's miles per gallon.
  • XGBoost: XGBRegressor - How to train a Regressor in XGBoost using a text based dataset, auto mpg, to predict a car's miles per gallon.

Containers

  • Keras: Sequential / Dense - How to train a Keras model using the Nightly Build of TensorFlow on AI Platform using a structured dataset, sonar signals, to predict whether the given sonar signals are bouncing off a metal cylinder or off a cylindrical rock.
  • PyTorch: Deep Neural Network - How to train a PyTorch model on AI Platform using a custom container with a image dataset, mnist, to classify handwritten digits.
  • PyTorch: Sequential - How to train a PyTorch model on AI Platform using a custom container with a structured dataset, sonar signals, to predict whether the given sonar signals are bouncing off a metal cylinder or off a cylindrical rock.
  • PyTorch: Sequential / HP Tuning - How to train a PyTorch model on AI Platform using a custom container and Hyperparameter Tuning with a structured dataset, sonar signals, to predict whether the given sonar signals are bouncing off a metal cylinder or off a cylindrical rock.

AI Platform Prediction (Online Predictions)

Notebook Tutorial:

  • scikit-learn: Model Serving - How to train a Random Forest Classifier in scikit-learn on your local machine using a text based dataset, Census, to predict a person’s income level and deploy it on AI Platform to create predictions.
  • XGBoost: Model Serving - How to train an XGBoost model on your local machine using a text based dataset, Census, to predict a person’s income level and deploy it on AI Platform to create predictions.

Complete Guide: Model Training and Prediction on AI Platform

Code Guide:

Hyperparameter Tuning (HP Tuning)

Code Guide:

Templates

Additional Resources

  • Cloud TPU

Please see the Cloud TPU guide for how to use Cloud TPU.

What do you want to see?

If you came looking for a sample we don’t have, please file an issue using the Sample / Feature Request template on this repository. Please provide as much detail as possible about the AI Platform sample you were looking for, what framework (Tensorflow, Keras, scikit-learn, XGBoost, PyTorch...), the type of model, and what kind of dataset you were hoping to use!

Jump below if you want to contribute and add that missing sample.

How to contribute?

We welcome external sample contributions! To learn more about contributing new samples, checkout our CONTRIBUTING.md guide. Please feel free to add new samples that are built in notebook form or code form with a README guide.

Want to contribute but don't have an idea? Check out our Sample Request Page and assign the issue to yourself so we know you're working on it!

Documentation

We host AI Platform documentation here