yanglei99/terraform_softlayer

Provision various clusters on Softlayer using Terraform, i.e.,Mesosphere DC/OS(GPU), Kubernetes, Slurm (GPU), LSF, Yarn (Spark, XGBoost, GPU), Ingestion(Spark, ZK, Kafka), Spark (XGBoost)... . And a generic Jupyter Notebook for Terraform Apply .

Terraform on Softlayer

Terraform Softlayer Provider

Notes on environment setup

Download Terraform binary and set onto PATH

Download Terraform Softlayer Provider binary and enable it in ~/.terraformrc. Make sure the binary is executable.

providers {
	softlayer = “/.../terraform-provider-softlayer_xxxx"
}

Scenarios

Basic Scenario

• VM. Provision 2 VMs with a new SSH Key
• Auto Scale Group. Provision Auto Scale Group and reuse the created SSH Key
• File Storage. Provision File Storage and mount as NFS to newly provisioned VMs with a new SSH Key

Data Center Platform

• Mesosphere DC/OS. Provision Mesosphere DC/OS cluster
• Kubernetes. Provision Kubernetes cluster
• Slurm. Provision Slurm cluster
• Yarn. Provision Yarn cluster. With optional add-on Spark and XGBoost.
• LSF. Provision IBM Spectrum Scale LSF cluster.

Specific Cluster

• XGBoost Scenario. Provision XGBoost Spark Cluster
• Data Ingestion Scenario. Provision Zookeeper, Spark, Kafka clusters for data ingestion.

Terraform UI

• Jupyter Notebook in Python3

Verified

• Terraform: 0.9.8
• Terraform Softlayer Provider: 1.5.1

Useful Terraform Command

terraform plan
terraform graph

terraform apply
terraform show

terraform plan --destroy
terraform destroy