We will go hands-on with an image classification of Cats vs Dogs demo using transfer learning, while leveraging MLflow to track our model experiments on Databricks.
You may find more information and a recording of this demo on the FourthBrain website.
If you would like run this demo, please download the two notebook files inside the notebooks
folder on Github:
1. Run them on Databricks Community Edition (free).
Note that Community Edition is a free offering of Databricks and there are a few features that are not available on Community Edition, such as the Model Registry (optional code at the end of "Transfer Learning Demo Part 2.ipynb").
(a) First sign up for a free Community Edition account, please refer to this doc if you have questions. You can use this link to sign in if you have an existing account.
(b) Download this file "Transfer Learning Demo Part 1.ipynb" inside of the notebooks
folder on this repo, and import this .ipynb file into your user account using Databricks Community Edition. Refer to this doc under "import a notebook" section.
(c) Create a cluster under the "Compute" tab on the left, select Databricks runtime version 10.4 LTS ML
, and name the cluster test10.4ML
. Please ensure you are using 10.4 LTS ML
and not 10.4 LTS
as the ML runtime pre-installs many useful libraries, such as TensorFlow & MLflow. Refer to this doc about how to create a cluster. It may take a few minutes to start a cluster.
(d) Open the imported notebooks on Databricks from the workspace tab, open "Transfer Learning Demo Part 1" notebook, attach the test10.4ML
cluster to your notebook, then click "Run All" on the top to run the notebook. Refer to this doc under the section "Attach a notebook to a cluster".
(e) You can also try to import and open "Transfer Learning Demo Part 2" notebook similarly, attach it to the cluster test10.4ML
, and click "Run All". It may take several minutes to run this notebook.
(f) Important note: you must attach your notebook to a cluster before you can run it. If your cluster is terminated, you need to restart it or re-create a new cluster.
(g) You may check your MLflow experiments by clicking "Experiments" on the top.
2. Or locally (for example, Jupyter notebook):
- Please follow this link for tensorflow transfer learning tutorial.
- To use MLflow locally, you may need to add additional code and steps for configuration. These steps may include
pip install mlflow
to install MLflow locally,!mlflow ui
to view the MLflow UI, and!pkill -f gunicorn
to stop the UI. Note: it is an exercise for the users, so the full code is not provided here.