anishnagaraj/aks-terraform

Azure Kubernetes Service with Terraform :new: :wheel_of_dharma: :sailboat: :cloud:

This repository is actively maintained @ https://github.com/dwaiba/aks-terraform

Table of Contents (Azure Kubernetes Service with Terraform)

1. Create ServicePrincipal and Subscription ID
2. Install terraform locally
3. Automatic provisioning
   • All in one with docker azure-cli-python
     • kubeconfig
     • Sanity
     • Jenkins master
     • Tiller Server with Draft and Brigade Server
     • kube-prometheus-grafana
4. License
5. Terraform graph
6. Code of conduct
7. Todo
8. Manual stepped provisioning
   • Run Azure cli container and copy terraform binary along with id_rsa to it
   • Clone this repo in the azure-cli-python container
   • Fill in the variables.tf with default values
   • Terraform for aks
9. Reporting bugs
10. Patches and pull requests

Have Fun checking a 4x speed AKS creation via asciinema - 3 node cluster with required jenkins plugins, tiller, ingress controllers, brigade, prometheus-grafana, takes around 20 minutes on Azure.

Create ServicePrincipal and Subscription ID

docker run -ti docker4x/create-sp-azure aksadmin

Your access credentials ==================================================

AD ServicePrincipal App ID: xxxxxx

AD ServicePrincipal App Secret: xxxxxx

AD ServicePrincipal Tenant ID: xxxxxx

Install terraform locally

wget https://releases.hashicorp.com/terraform/0.11.7/terraform_0.11.7_linux_amd64.zip -O temp.zip; unzip temp.zip; rm temp.zip ;sudo cp terraform /usr/local/bin

Automatic provisioning

All in one with docker azure-cli-python

Please note docker should be installed with terraform binary and your id_rsa.pub present in directory for running the following.

Terraform locally installed has binary in /usr/local/bin

Create new cluster -Please note docker should be installed with terraform binary and your id_rsa.pub present in directory for running the following.

wget https://raw.githubusercontent.com/dwaiba/aks-terraform/master/create_cluster.sh && chmod +x create_cluster.sh && ./create_cluster.sh

Terraform will now prompt for the 10 variables as below in sequence:

• agent_count
• azure_container_registry_name
• client_id
• client_secret
• cluster_name
• dns_prefix
• helm_install_jenkins
• kube_version
• location
• resource_group_name

Values and conventions for the 10 variables are as follows :

• agent_count are the number of "agents" - 3 or 5 or 7
• azure_container_registry_name as "alphanumeric"
• client_id which is the sp client Id
• client_secret which is the secret for the above as created in pre-req
• cluster_name as "--org--aks--yournameorBU--"
• dns_prefix as "--org--aks--yournameorBU--"
• helm_install_jenkins is either true or false
• kube_version may vary from 1.9.x to 1.11.1 through 10.3.6- Please note 1.11.1 is only available in the American regions
• location of the resource group and is dependant on the version above. - westeurope (10.3.6) or eastus(1.11.1)
  • Please Azure Service Availability for AKS in Regions and also via az aks get-versions --location
• resource_group_name as "--org--aks--yournameorBU--"

The DNSPrefix must contain between 3 and 45 characters and can contain only letters, numbers, and hyphens. It must start with a letter and must end with a letter or a number.

Only alpha numeric characters only are allowed in azure_container_registry_name.

Expected account_tier for storage to be one of Standard Premium with max GRS and not RAGRS. storage_account_id can only be specified for a Classic (unmanaged) Sku of Azure Container Registry. This does not support web hooks. Default is Premium Sku of Azure Container Registry.

KUBECONFIG

echo "$(terraform output kube_config)" > ~/.kube/azurek8s

Also one can echo and copy content to local kubectl config.

export KUBECONFIG=~/.kube/azurek8s

Sanity

kubectl get nodes

kubectl proxy

Dashboard available at http://localhost:8001/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/#!/overview?namespace=default.

or if proxied from a server can be online as follows:

kubectl proxy --address 0.0.0.0 --accept-hosts .* &

Jenkins Master

After Cluster creation all you need to do is perform "kubectl get svc" to get url for jenkins and obtain jenkins password as follows- preferably from within the container prompt post creation:

printf $(kubectl get secret --namespace default hclaks-jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 -d);echo

Tiller Server with Draft and Brigade Server

Auto Provisioned.

One can also use draft with the Container Registry and use helm to install any chart as follows:

kube-prometheus-grafana

Provisioned by master main.tf local-exe provisioner via git clone https://github.com/coreos/prometheus-operator.git without RBAC- global.rbacEnable=false and without prometheus-operator .

Dashboard available post port forward via:

kubectl get pods --namespace monitoring

kubectl get pods kube-prometheus-grafana-6f8554f575-bln7x --template='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' --namespace monitoring

kubectl port-forward kube-prometheus-grafana-6f8554f575-bln7x 3000:3000 --namespace monitoring &

User/Password for grafana (generally admin/admin)

kubectl get secret --namespace monitoring kube-prometheus-grafana -o jsonpath="{.data.password}" | base64 --d ; echo

kubectl get secret --namespace monitoring kube-prometheus-grafana -o jsonpath="{.data.user}" | base64 --d ; echo

License

• Please see the LICENSE file for licensing information.

Code of Conduct

• Please see the Code of Conduct

Terraform Graph

Please generate dot format (Graphviz) terraform configuration graphs for visual representation of the repo.

terraform graph | dot -Tsvg > graph.svg

Attached is the present master Branch graph. (Click to enlarge)

Also, one can use Blast Radius on live initialized terraform project to view graph. A live example is here for this project. A picture is attached below on master. Blast Radius is a pip3 install.

Todo

• RBAC
• Service Mesh
• Kashti

Manual stepped provisioning

Run Azure cli container and copy terraform binary along with id_rsa to it

docker run -dti --name=azurecli-python --restart=always azuresdk/azure-cli-python && docker cp terraform azure-cli-python:/ && docker cp ~/.ssh/id_rsa azure-cli-python:/ && docker exec -ti azure-cli-python bash -c "az login && bash"

Clone this repo in the azure-cli-python container

git clone https://github.com/dwaiba/aks-terraform

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl;

Optionally, you can also install kubectl locally. This repo installs kubectl in the azure-cli-python container.

chmod +x ./kubectl;

mv ./kubectl /usr/local/bin/kubectl;

mv /id_rsa.pub /aks-terraform;

Fill in the variables file with default values

Terraform for aks

mv ~/terraform aks-terraform/ cd aks-terraform terraform init

terraform plan -out run.plan

terraform apply "run.plan"

Reporting bugs

Please report bugs by opening an issue in the GitHub Issue Tracker. Bugs have auto template defined. Please view it here

Patches and pull requests

Patches can be submitted as GitHub pull requests. If using GitHub please make sure your branch applies to the current master as a 'fast forward' merge (i.e. without creating a merge commit). Use the git rebase command to update your branch to the current master if necessary.

Contributors

✨ Recognize all contributors, not just the ones who push code ✨

Thanks goes to these wonderful people :

anishnagaraj	Ranjith	cvakumark	Dwai Banerjee

This project follows the all-contributors specification. Contributions of any kind welcome!