/learn_modeldb

Primary LanguageJupyter Notebook

Prerequisites

  • Python requirements (python -m pip install -r requirements.txt)
  • ModelDB (docker-compose -f docker-compose-all.yaml pull at this repository's root)
  • Docker
  • Jenkins
  1. Run ModelDB.
  2. Run Jenkins (jenkins-lts --httpPort=7070, since ModelDB occupies port 8080).
  3. Run Jupyter (jupyter notebook).

Ad Hoc

The ad hoc workflow logs model ingredients to S3 and fetches them, one by one, to deploy them. This can lead to mismatched artifacts if S3 buckets are not carefully managed, and it can be difficult to track what results are associated with what model.

  1. Run the notebook, logging model ingredients to S3.
  2. Run the Jenkins pipeline, building a Docker image to serve the model.
    1. Or run 02-package/run.sh directly, setting the environment variables BUCKET, MODEL_PATH, and METADATA_PATH to point at the model ingredients on S3.
  3. Run 03-predict/run.sh to serve the model.
    1. Run 03-predict/predict.sh to make predictions against the model at localhost:5000.
    2. View live metrics from the model in your web browser at http://localhost:9090/.

ModelDB Versioning

The ModelDB Versioning workflow leverages our versioning system to snapshot model ingredients together, linking them to experimental results and enabling reproducibility, reverts, and merges of promising ingredients.

  1. Run the notebooks, logging model ingredients to S3.
  2. Run the Jenkins pipeline, building a Docker image to serve the model.
    1. Or run 02-package/run.sh directly, setting the environment variables VERTA_HOST and RUN_ID to fetch the associated model ingredients from ModelDB.
  3. Run 03-predict/run.sh to serve the model.
    1. Run 03-predict/predict.sh to make predictions against the model at localhost:5000.
      • Try a few German phrases, as well! 
        [
    "Guten Morgen.",               # Good morning.
    "Gute Nacht.",                 # Good night!
    "Sie sind sehr freundlich.",   # You're very kind!

    "Da muss ich widersprechen.",  # I disagree.
    "Es ist ein Notfall!",         # It's an emergency!
    "Ich verstehe nicht.",         # I don't understand.
    "Ich bin sauer.",              # I'm angry.
]
    2. View live metrics from the model in your web browser at http://localhost:9090/.