ZenML Implementation Guide

Overview

This guide outlines the step-by-step process for setting up and running the demonstrated ZenML pipeline with Neptune experiment tracking integration. The implementation follows a systematic approach to ensure reproducible machine learning workflows.

Prerequisites

Python 3.9 or higher
Access to Neptune.ai account
ZenML cloud account

Installation and Setup Process

1. Environment Setup

First, create and activate a dedicated virtual environment:

# Create virtual environment
python -m venv .venv

# Activate virtual environment
# For Unix/MacOS
source .venv/bin/activate

2. Dependencies Installation

Install required packages from the requirements file:

pip install -r requirements.txt

3. ZenML Configuration

Initialize and configure ZenML with the following steps:

# Initialize ZenML in your project directory
zenml init
zenml integration install pytorch_lightning neptune

# Connect to ZenML cloud tenant (you can find this command in the overview page of your ZenML cloud tenant)
zenml login 8a462fb6-b...

# Register Neptune experiment tracker
zenml experiment-tracker register neptune_experiment_tracker \
    --flavor=neptune \
    --project="" \
    --api_token=""

# Register and configure stack
zenml stack register neptune_stack \
    -o default \
    -a default \
    -e neptune_experiment_tracker

# Set as active stack
zenml stack set neptune_stack

4. Execute Pipeline

Run the implementation:

python run.py

Troubleshooting

Ensure all environment variables are properly set
Verify Neptune.ai credentials are correctly configured
Check ZenML stack status using zenml stack list

zenml-io/cifar10-resnet-demo