/terraform-aws-eks-jupyterhub

Running JupyterHub on Kubernetes (AWS EKS) in 30 minutes :fire:

Primary LanguageHCL

terraform-eks-jupyterhub


Running Jupyter Hub on AWS EKS using Terraform in 30 minutes

Requirements

Generate your ssh key named aws_infra_root_key without password and import it into AWS

# Install AWS CLI - Do not install in virtualenv
pip install --upgrade awscli

# configure AWS credential
aws configure

# Generate SSH Key (without password - input enter)
mkdir ~/.ssh || true 
cd ~/.ssh
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Enter a file in which to save the key (/Users/you/.ssh/id_rsa): aws_infra_root_key

# Import SSH Public Key into AWS
aws ec2 import-key-pair --key-name "aws_infra_root_key" --public-key-material file://~/.ssh/aws_infra_root_key.pub

Install required CLI tools.

# OSX commands
brew install aws-iam-authenticator
brew install terraform

Then prepare AWS resources to apply terraform projects.

  • Create S3 bucket named terraform-infra-{SOMETHING}
  • Create DynamoDB table named terraform-lock-resource and terraform-lock-iam
    • with primary key LockID (String)
  • Modify terraform backend S3 bucket name to terraform-infra-{SOMETHING}
    • in terraform.tf and data.tf

Project Structure

├── k8s-jupyter-nbviewer     # Kubernetes Manifest for NbViewer 
├── k8s-jupyter-hub          # Kubernetes Manifest for Jupyter Hub

├── k8s-system-autoscaler    # Kubernetes Manifest for Cluster Autoscaler 
├── k8s-system-dashboard     # Kubernetes Manifest for Kuberntes Dashboard 
├── k8s-system-elasticstack  # Kubernetes Manifest for ELK Stack 
├── k8s-system-helm          # Kubernetes Manifest for Helm 
├── k8s-system-prometheus    # Kubernetes Manifest for Prometheus 
├── k8s-system-reloader      # Kubernetes Manifest for Reloader 

├── terraform-root-vpc       # Terraform Project for VPC (Network)
├── terraform-root-iam       # Terraform Project for IAM (Role, Permissions)
├── terraform-root-bastion   # Terraform Project for Bastion (EC2)
└── terraform-root-eks       # Terraform Project for EKS (Kubernetes)

Terraforming AWS Infrastructure

terraform-root-vpc

$ terraform init
$ terraform plan

$ terraform apply

terraform-root-iam

$ terraform init
$ terraform plan

$ terraform apply

terraform-root-eks

$ terraform init
$ terraform plan

$ terraform apply

$ aws eks update-kubeconfig --name analysis-production

$ kubectl get nodes

NAME                                                STATUS    ROLES     AGE       VERSION
ip-10-XXX-XXX.XXX.ap-northeast-2.compute.internal   Ready     <none>    2m15s     v1.13.7-eks-c57ff8
ip-10-YYY-YYY-YYY.ap-northeast-2.compute.internal   Ready     <none>    2m15s     v1.13.7-eks-c57ff8

terraform-root-bastion

after EKS apply

$ terraform init
$ terraform plan

$ terraform apply

Setup Kubernetes Applications

Resource Destroy

terraform-root-bastion

$ terraform destroy

terraform-root-eks

$ terraform destroy

terraform-root-iam

$ terraform destroy

terraform-root-vpc

$ terraform destroy

References

Credits