Dolly Model Register on Mflow and Serve using Databricks Model Serving

Question

Dolly Model Register on Mflow and Serve using Databricks Model Serving

SrikarPrayaga06 opened this issue 2 years ago · 1 comments

I am trying to register the dolly model on mlflow and serve it using the databricks model serving. The serving container keeps failing and endpoint is unable to be created.

Answer 1 · 2023-06-06T22:59:59.000Z

You won't be able to run this in Databricks serving unless you are opted into the GPU serving private preview. The preview should let you serve the 7B model. (This is a Databricks question so I'll close it here)