Azure Mini Project

Databricks with Walmart dataset

Goals

In this mini-project, you will learn how to:

  • Deploy a Databricks cluster.
  • Work with content stored in Azure Blob Storage and accessed by the Spark cluster as an HDFS volume.
  • Use a Jupyter Notebook to interactively explore a large dataset.
  • Delete a cluster to avoid incurring unnecessary charges.

Step 1: upload file to Azure Blob Storage using upload_file.sh

Step 2: upload Jupyter Notebook to Databricks cluster