/CI-CD-Pipeline-Optimisation

A CI/CD Pipeline optimised with IaC and automation

Primary LanguageHCL

CONTINUOUS INTEGRATION AND CONTINUOUS DEPLOYMENT (CI/CD) PIPELINE OPTIMISATION

This project focuses on automating the continuous integration and continuous deployment (CI/CD) of a web application using Jenkins and Docker. The goal is to streamline the development process, reduce manual intervention, and ensure efficient and reliable application delivery.

DEVOPS LIFECYCLE

ARCHITECTURAL DIAGRAM

TECHNOLOGIES & TOOLS

  • Planning (Jira)
  • Code Writing (Visual Studio Code)
  • Web Application Framework (Python (Flask))
  • Version Control (Git)
  • Build Automation (Jenkins)
  • Artifact Storage (JFrog Artifactory)
  • Containerisation (Docker)
  • Infrastructure as Code (IaC) (Terraform)
  • Integration Testing Framework (pytest)
  • Cloud Service (Amazon Web Services (AWS))
  • Release Manager (GitHub Actions)
  • Monitoring & Logging (Prometheus and Grafana)

AWS SERVICES

  • Amazon EC2 - Computing Resources
  • S3 - Storage
  • Elastic Load Balancer - Distributing incoming traffic
  • Autoscaling - Auto increase number of instances running app

TASKS

  1. Web Application Setup: Create a simple web application using Flask web framework
  2. Version Control Setup: Initialize a Git repository for the web application and set up a remote repository (on GitHub).
  3. Jenkins Setup: Install and configure Jenkins on a server or cloud instance.
  4. Create Jenkins Jobs: Create Jenkins jobs for the CI/CD pipeline, including build, test, and deployment stages.
  5. Automate Build Stage: Write a Jenkinsfile (or use the declarative pipeline syntax) to automate the build process. This could include compiling code, installing dependencies, and creating a deployable artifact.
  6. Automated Testing: Integrate automated testing (unit and integration tests) into the pipeline. Use the appropriate testing frameworks and ensure that tests are executed automatically.
  7. Dockerization: Create a Dockerfile to containerize the web application. Build and push Docker images to a container registry.
  8. Automate Deployment: Write deployment scripts to deploy the web application using Docker containers. Ensure that deployments can be triggered automatically from the CI/CD pipeline.
  9. Monitoring and Logging: Implement monitoring and logging using Docker logs and monitoring tools (e.g., Prometheus and Grafana).
  10. Documentation and Reporting: Create documentation that outlines the CI/CD pipeline's architecture and how to trigger builds, tests, and deployments. Generate reports on the pipeline's performance.
  11. CI/CD Pipeline Optimization: Continuously monitor the CI/CD pipeline's performance, identify bottlenecks, and optimize the workflow for faster and more reliable delivery.

STEPS

  1. Set up the web application

  2. Set up Version Control

  3. Set up Jenkins

    • Install Github Integration Plugin in Jenkins in order for Jenkins to work with our GitHub repo.
    • Configure webhook -Navigate to project repository on Github > webhooks
      • Click "add webhook"
      • Configure the following:
        • Payload URL: https://f475-82-3-193-54.ngrok.io/ghprbhook/ ngrok.exe http (e.g. 80)
        • Content type: application/json
        • Secret:
        • Which events would you like to trigger this webhook?: Choose "Let me select individual events" Select "Pull Requests" and "Pushes"
        • Make sure "Active is checked/ticked
      • Add webhook
    • Create Jenkins project
      • In Jenkins, click 'New Item' to create a new project.
      • Enter item name as 'CICDPipelineOptimisation'
      • Chose 'Pipeline' as project
      • Create Project
      • Configure Pipeline
        • Enter 'A CI/CD Pipeline' as the description
        • Tick 'Github Project' and enter the project url
        • Configure Build Triggers
          • Tick 'GitHub hook trigger for GITSCM polling' to allow Jenkins work whenever a push is made to GitHub
          • Tick 'Poll SCM', and enter 'H/2 * * * * ' as Schedule. To poll every 2 minutes
        • Configure pipeline
          • Set 'Pipeline script from SCM' as Definition
          • Choose Git as SCM
          • Enter the repositories details
          • Enter Jenkinsfile as Script Path
        • Save

  4. Set up Artifactory (to store build artifacts)

    • Install and Initial Setup
      • Download on local machine (windows) from https://tinyurl.com/yk8mv856 *this link has been shortened
      • Extract All
      • Move extracted folder to This PC
      • Navigate into the folder and select 'activate.bat'
      • Artifactory will be installed
      • Navigate to http://localhost:8081/artifactory/webapp/
      • Login with the following credentials
        • username: admin
        • password: password
      • Once logged in, a modal window will show, and follow through the following steps:
        • Set Admin Password
          • Set a new password
        • Skip the 'Configure a Proxy Server'
        • Create repositories
          • Choose 'Generic' as the package type
          • Click 'Create
        • Finish
    • Set up another user
      • Navigate to Admin on the menu
      • Under Security, Select 'Users'
      • Add New User
        • Enter Username
        • Enter Email Address
        • Give the user 'Admin Privileges'
        • Set password
        • Save

  5. Configure Jenkins

    • Navigate to manage Jenkins -> Plugins -> Available plugins
      • Search for 'Artifactory', select, and Install without restart
    • Navigate to Manage Jenkins -> System
      • Scroll down to JFrog
      • Click 'Add JFrog Platform Instance' and enter the required details:
        • Instance ID: my-jfrog-server
        • URL: http://localhost:8081/ *if on localhost
        • Default Deployer Credentials
          • Username and Password of the new user created
      • Test Connection
    • Save Changes

  6. Automate Build Stage

  7. Store Artifact

  8. Set up Development Environment

  9. Set up Testing Environment

SIMULATION

RESULTS