christones/CapstoneProjectforDevOps

In this project i will apply the skills and knowledge which were developed throughout the Cloud DevOps Nanodegree program. These include: Working in AWS Using Jenkins or Circle CI to implement Continuous Integration and Continuous Deployment Building pipelines Working with Ansible and CloudFormation to deploy clusters Building Kubernetes clusters Building Docker containers in pipelines

CapstoneProjectforDevOps-Udacity

This is the Capstone project for Udacity's "Cloud DevOps Engineer" Nanodegree Program. In this project i will apply the skills and knowledge which were developed throughout the Cloud DevOps Nanodegree program. These include:

• Working in AWS
• Using Jenkins to implement Continuous Integration and Continuous Deployment
• Building pipelines
• Working with Ansible and CloudFormation to deploy clusters
• Building Kubernetes clusters
• Building Docker containers in pipelines

---

Tools Used

• Git & GitHub
• AWS & AWS-CLI
• Python3
• Flask framework.
• pip3
• Pylint
• Docker & Docker-Hub Registery
• Jenkins
• Kubernetes CLI (kubectl)
• EKS
• CloudFormation
• BASH
• LucidChart

---

Project Steps

1. Development
2. Jenkins
3. Setup Kubernetes Cluster
4. CI/CD Pipeline
5. Cost of Greatness

---

Development

• Simple flask application.

---

• Docker Containerization (Local manual check):

  Run docker flask-app container:

  sudo docker build -t flask-app .

  

  sudo docker run -d -p 5000:5000 --name flask-app flask-app

   A string logs will be generated after execution
  

  sudo docker logs “generatedlogs”

   Example : sudo docker logs  fbd1cac28611dcae10b9f8fd17af2cd8522418c77d98f8254ea4bad90820527c
  

• Push docker image to docker-hub (Local manual check):

  Pushing to the registry :

  sudo docker tag flask-app:latest christones/flask-app:latest

  sudo docker login --username=christones

  sudo docker push christones/flask-app:latest

  

---

Jenkins

• Create security-group for jenkins:

• Create jenkins EC2:

Starting with Choosing the Amazon Machine Image OS

Choosing an instance type

Configuring security groups

Launching the instance

• Connect to jenkins ec2:

  chmod 400  Udacity-CapstonebyChris.pem (Activating the downloaded Key Pair RSA signature )
  
  ssh -i "Udacity-CapstonebyChris.pem" ec2-user@ec2-44-201-195-107.compute-1.amazonaws.com
  
  
  

• 

• Setup Jenkins Server:

  • Install java:

    $ sudo yum update && sudo yum install java-1.8.0-openjdk
    

  • Install Jenkins.

  $ curl --silent --location http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo | sudo tee /etc/yum.repos.d/jenkins.repo
  
  $ sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
  
  $ cat /etc/yum.repos.d/jenkins.repo  (Verifying the download)
  
  $ sudo yum install jenkins 
  
  

  • Install pip3 and venv:

    $ sudo yum install python3-pip
    

    $ spython3 -m venv python3-virtualenv
    

  • Install "Blue-Ocean-Aggregator" Plug-In.

  

• Docker With Jenkins:

  • Install docker on jenkins server.

  • Add jenkins to docker group:

    $ sudo usermod -aG docker jenkins
    

  • Install "Docker" jenkin's plug-in.

  • Add Docker-Hub credentials to jenkins.

  • Use docker plug-in to build, upload, and delete docker images.

• AWS With Jenkins:

  • Install "Pipeline-AWS" Plug-In.
  • Add AWS-User credentials to jenkins.

• Kubernetes With Jenkins:

  • Install kubectl.

---

Setup Kubernetes Cluster

Create kubernetes "Production" Cluster on AWS using EKS: (From my local machine)

• Useful resource here .

• Install AWS CLI.

• Install eksctl.

• Install kubectl.

• Create Amazon EKS cluster:

  1. Create an AWS IAM service role:

  

  2. Create Network (VPC,Subnets,SecurityGroups,InternetGateway,RouteTables) to deploy the cluster using CloudFormation/amazon-eks-vpc-sample.yaml

  

  

  

  3. Create AWS EKS Cluster:

  

  4. Configure kubectl for Amazon EKS:

  $ aws eks --region us-east-2 update-kubeconfig --name production
  

  kubectl config current-context
  

  

  5. Create worker nodes to join kubernetes cluster using CloudFormation/amazon-eks-nodegroup.yaml:

  

  

  6. Enable the worker nodes to join cluster using k8s/aws-auth-cm.yaml:

  kubectl apply -f ~/.kube/aws-auth-cm.yaml
  

  check nodes :

  kubectl get nodes
  

  

  

  7. Test deploying flask-app on the production cluster outside pipeline:

  kubectl apply -f k8s/blue-deployment.yaml 
  

  kubectl apply -f k8s/service.yaml 
  

  kubectl get all
  

  

  Access the app from browser:

  

---

CI/CD Pipeline

Overview:

Steps:

1. Install needed packages from requirements.txt.

2. Linting Code:

3. Set K8S Context: To enable jenkins to run kubectl commands with "aws-user" credentials stored in jenkins server.

4. Build Green Docker Image.

5. Push green image (mahaamin97/pre-production-flask-app) to docker-hub registery:

   • Link to pre-production-flask-app Image

6. Clean Up green image: delete pre-production-flask-app Image from jenkins server after uploading it to docker-hub, to save jenkin's server disk space.

7. Blue/Green Deployment Demonstration:

   • Blue --> production deployment (flask-app)

   • Green --> pre-production deployment (pre-production-flask-app)

   • flask-app-svc --> main service endpoint.

   • test-svc --> service on green deployment only for testing purposes.

   • If green deployment succeeded :

     • switch traffic to green deployment

     • changes are deployed to blue deployment (pipeline ends having two identical environments)

     • switch back service to blue deployment

   • Green deployment succeeded:

   

   Green and Blue environments are the same (until new commit happens)

   

   • Else if Green deployment failed, the main service (flask-app-svc) still points to blue deployment, while green deployment changed and can be accessed via test-svc:

   

   

8. Test Green Deployment:

9. Blue Docker Image:

   • Link to flask-app Image on docker-hub.

---

Cost of Greatness

• Final Jenkins Dashboard:

• AWS Billing:

---