/airflow_orchestration_ml_pipeline

Primary LanguagePython

Machine Learning Pipeline using Apache Airflow

Airflow Logo

This repository contains a simple Apache Airflow pipeline designed to process data, train a machine learning model, and test it using Airflow orchestration. The pipeline is Dockerized for easy deployment and reproducibility.

Prerequisites

Before getting started, make sure you have Docker installed on your machine.

Quick Start

To run the machine learning pipeline, follow these steps:

  1. Start the Airflow service:

    airflow compose up --build

  2. Access the Airflow UI at http://localhost:8080/.

Image 1

  1. Log in using the following credentials:

    • Username: airflow
    • Password: airflow

  2. Trigger your desired pipeline from the Airflow dashboard.

Image 2

Stopping the Service

To stop the Airflow service and remove the containers, run:

airflow compose down