MahaAmin/Udacity-DevOps-Capstone

Udacity's cloud devops nanodegree capstone project

Udacity Capstone Project

Project Overview

Capstone project for Udacity's "Cloud DevOps Engineer" Nanodegree Program.

---

Objectives

• Working in AWS
• Using Jenkins to implement Continuous Integration and Continuous Deployment
• Building pipelines
• Working with Ansible and CloudFormation to deploy clusters
• Building Kubernetes clusters
• Building Docker containers in pipelines

---

Tools Used

• Git & GitHub
• AWS & AWS-CLI
• Python3
• Flask framework.
• pip3
• Pylint
• Docker & Docker-Hub Registery
• Jenkins
• Kubernetes CLI (kubectl)
• EKS
• CloudFormation
• BASH
• LucidChart

---

Project Steps

1. Development
2. Jenkins
3. Setup Kubernetes Cluster
4. CI/CD Pipeline
5. Cost of Greatness

---

Development

• Simple flask application.

---

• Docker Containerization (Local manual check):

  Run docker flask-app container:

      $./run_docker.sh
  

• Push docker image to docker-hub (Local manual check):

      $./upload_docker.sh
  

---

Jenkins

• Create security-group for jenkins:

• Create jenkins EC2:

• Connect to jenkins ec2:

  ssh -i udacity-capstone.pem ubuntu@ec2-18-220-188-146.us-east-2.compute.amazonaws.com
  

• Setup Jenkins Server:

  • Install java:

    $ sudo apt update && sudo apt install default-jdk;
    

  • Install Jenkins.

  • Install pip3 and venv:

    $ sudo apt install python3-pip
    

    $ sudo apt-get install python3-venv
    

  • Install "Blue-Ocean-Aggregator" Plug-In.

  

• Docker With Jenkins:

  • Install docker on jenkins server.

  • Add jenkins to docker group:

    $ sudo usermod -aG docker jenkins
    

  • Install "Docker" jenkin's plug-in.

  • Add Docker-Hub credentials to jenkins.

  • Use docker plug-in to build, upload, and delete docker images.

• AWS With Jenkins:

  • Install "Pipeline-AWS" Plug-In.
  • Add AWS-User credentials to jenkins.

• Kubernetes With Jenkins:

  • Install kubectl.

---

Setup Kubernetes Cluster

Create kubernetes "Production" Cluster on AWS using EKS: (From my local machine)

• Useful resource here .

• Install AWS CLI.

• Install eksctl.

• Install kubectl.

• Create Amazon EKS cluster:

  1. Create an AWS IAM service role:

  

  2. Create Network (VPC,Subnets,SecurityGroups,InternetGateway,RouteTables) to deploy the cluster using CloudFormation/amazon-eks-vpc-sample.yaml

  

  

  

  3. Create AWS EKS Cluster:

  

  4. Configure kubectl for Amazon EKS:

  $ aws eks --region us-east-2 update-kubeconfig --name production
  

  kubectl config current-context
  

  

  5. Create worker nodes to join kubernetes cluster using CloudFormation/amazon-eks-nodegroup.yaml:

  

  

  6. Enable the worker nodes to join cluster using k8s/aws-auth-cm.yaml:

  kubectl apply -f ~/.kube/aws-auth-cm.yaml
  

  check nodes :

  kubectl get nodes
  

  

  

  7. Test deploying flask-app on the production cluster outside pipeline:

  kubectl apply -f k8s/blue-deployment.yaml 
  

  kubectl apply -f k8s/service.yaml 
  

  kubectl get all
  

  

  Access the app from browser:

  

---

CI/CD Pipeline

Overview:

Steps:

1. Install needed packages from requirements.txt.

2. Linting Code:

3. Set K8S Context: To enable jenkins to run kubectl commands with "aws-user" credentials stored in jenkins server.

4. Build Green Docker Image.

5. Push green image (mahaamin97/pre-production-flask-app) to docker-hub registery:

   • Link to pre-production-flask-app Image

6. Clean Up green image: delete pre-production-flask-app Image from jenkins server after uploading it to docker-hub, to save jenkin's server disk space.

7. Blue/Green Deployment Demonstration:

   • Blue --> production deployment (flask-app)

   • Green --> pre-production deployment (pre-production-flask-app)

   • flask-app-svc --> main service endpoint.

   • test-svc --> service on green deployment only for testing purposes.

   • If green deployment succeeded :

     • switch traffic to green deployment

     • changes are deployed to blue deployment (pipeline ends having two identical environments)

     • switch back service to blue deployment

   • Green deployment succeeded:

   

   Green and Blue environments are the same (until new commit happens)

   

   • Else if Green deployment failed, the main service (flask-app-svc) still points to blue deployment, while green deployment changed and can be accessed via test-svc:

   

   

8. Test Green Deployment:

9. Blue Docker Image:

   • Link to flask-app Image on docker-hub.

---

Cost of Greatness

• Final Jenkins Dashboard:

• AWS Billing:

---