/Udacity-DevOps-Capstone

Udacity's cloud devops nanodegree capstone project

Primary LanguageShell

Udacity Capstone Project

Project Overview

Capstone project for Udacity's "Cloud DevOps Engineer" Nanodegree Program.


Objectives

  • Working in AWS
  • Using Jenkins to implement Continuous Integration and Continuous Deployment
  • Building pipelines
  • Working with Ansible and CloudFormation to deploy clusters
  • Building Kubernetes clusters
  • Building Docker containers in pipelines

Tools Used

  • Git & GitHub
  • AWS & AWS-CLI
  • Python3
  • Flask framework.
  • pip3
  • Pylint
  • Docker & Docker-Hub Registery
  • Jenkins
  • Kubernetes CLI (kubectl)
  • EKS
  • CloudFormation
  • BASH
  • LucidChart

Project Steps

  1. Development
  2. Jenkins
  3. Setup Kubernetes Cluster
  4. CI/CD Pipeline
  5. Cost of Greatness

Development

  • Simple flask application.

  • Docker Containerization (Local manual check):

    Run docker flask-app container:

        $./run_docker.sh
    

  • Push docker image to docker-hub (Local manual check):

        $./upload_docker.sh
    

Jenkins

  • Create security-group for jenkins:

1-jenkins-sg.png

  • Create jenkins EC2:

2-jenkins-ec2.png

  • Connect to jenkins ec2:

    ssh -i udacity-capstone.pem ubuntu@ec2-18-220-188-146.us-east-2.compute.amazonaws.com
    
  • Setup Jenkins Server:

    • Install java:

      $ sudo apt update && sudo apt install default-jdk;
      
    • Install Jenkins.

    • Install pip3 and venv:

      $ sudo apt install python3-pip
      
      $ sudo apt-get install python3-venv
      
    • Install "Blue-Ocean-Aggregator" Plug-In.

    3-jenkins-blueocean.png

  • Docker With Jenkins:

    • Install docker on jenkins server.

    • Add jenkins to docker group:

      $ sudo usermod -aG docker jenkins
      
    • Install "Docker" jenkin's plug-in.

    • Add Docker-Hub credentials to jenkins.

    • Use docker plug-in to build, upload, and delete docker images.

4-jenkins-credentials.png

  • AWS With Jenkins:

    • Install "Pipeline-AWS" Plug-In.
    • Add AWS-User credentials to jenkins.
  • Kubernetes With Jenkins:

    • Install kubectl.

Setup Kubernetes Cluster

Create kubernetes "Production" Cluster on AWS using EKS: (From my local machine)

  • Useful resource here .

  • Install AWS CLI.

  • Install eksctl.

  • Install kubectl.

  • Create Amazon EKS cluster:

    1. Create an AWS IAM service role:

    5-eks-service-iam-role.png

    1. Create Network (VPC,Subnets,SecurityGroups,InternetGateway,RouteTables) to deploy the cluster using CloudFormation/amazon-eks-vpc-sample.yaml

    6-eks-vpc.png

    6.1-vpc-for-eks-stack.png

    7-vpc-for-eks-resources.png

    1. Create AWS EKS Cluster:

    7.1-eks-cluster-production.png

    1. Configure kubectl for Amazon EKS:
    $ aws eks --region us-east-2 update-kubeconfig --name production
    
    kubectl config current-context
    

    8-kubectl-config-current-context.png

    1. Create worker nodes to join kubernetes cluster using CloudFormation/amazon-eks-nodegroup.yaml:

    9-eks-groupnode-stack.png

    10-eks-groupnode-resources.png

    1. Enable the worker nodes to join cluster using k8s/aws-auth-cm.yaml:
    kubectl apply -f ~/.kube/aws-auth-cm.yaml
    

    check nodes :

    kubectl get nodes
    

    11-kubectl-get-nodes.png

    12-node-groups-ec2s.png

    1. Test deploying flask-app on the production cluster outside pipeline:
    kubectl apply -f k8s/blue-deployment.yaml 
    
    kubectl apply -f k8s/service.yaml 
    
    kubectl get all
    

    13-kubectl-get-all.png

    Access the app from browser:

    14-app-in-browser.png


CI/CD Pipeline

Overview:

15-Jenkins-Pipeline.png

Steps:

  1. Install needed packages from requirements.txt.

  2. Linting Code:

16-lint-failed.png

17-lint-success.png

  1. Set K8S Context: To enable jenkins to run kubectl commands with "aws-user" credentials stored in jenkins server.

  2. Build Green Docker Image.

  3. Push green image (mahaamin97/pre-production-flask-app) to docker-hub registery:

18-docker-hub.png

  1. Clean Up green image: delete pre-production-flask-app Image from jenkins server after uploading it to docker-hub, to save jenkin's server disk space.

  2. Blue/Green Deployment Demonstration:

    • Blue --> production deployment (flask-app)

    • Green --> pre-production deployment (pre-production-flask-app)

    • flask-app-svc --> main service endpoint.

    • test-svc --> service on green deployment only for testing purposes.

    • If green deployment succeeded :

      • switch traffic to green deployment

      • changes are deployed to blue deployment (pipeline ends having two identical environments)

      • switch back service to blue deployment

    • Green deployment succeeded:

    19-blue-deployment.png

    Green and Blue environments are the same (until new commit happens)

    20-blue-green-both-blue.png

    • Else if Green deployment failed, the main service (flask-app-svc) still points to blue deployment, while green deployment changed and can be accessed via test-svc:

    21-green-deployment-failed.png

    22-blue-green-failed.png

  3. Test Green Deployment:

23-test-green-deployment.png

  1. Blue Docker Image:


Cost of Greatness

  • Final Jenkins Dashboard:

24-jenkins-dashboard.png

  • AWS Billing:

25-aws-billing.png