Find a home for HiPPO

Question

Find a home for HiPPO

Closed this issue a year ago · 5 comments

erskine commented a year ago

Answer 1 · 2023-08-25T21:01:41.000Z

RTFM, but the gist is likely to be:

Pick a instance size G4DNXL (smallest GPU)
Create a job that listens to slack web socket
Script up the dependencies for the default ML runtime (pip installs, etc.)
Figure out where to put the model for invocation (ideally MLFlow)

Answer 2 · 2023-08-26T04:59:58.000Z

Unfortunately, the newest databricks runtime 13.3 LTS ML uses a cuda version of 11.4 (via nvidia-smi), but the earliest installation allowed by MLC is 11.6.

Fortunately, databricks released a blog about how to run llama2. https://www.databricks.com/blog/building-your-generative-ai-apps-metas-llama-2-and-databricks

It hasn't worked perfectly out of the box, but I'll keep cracking at it.

OSError: meta-llama/Llama-2-7b-chat-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with use_auth_token or log in with huggingface-cli login and pass use_auth_token=True.

Answer 3 · 2023-08-26T20:06:47.000Z

Create a huggingface 🤗 account: https://huggingface.co/join
Request access to Llama 2: https://ai.meta.com/resources/models-and-libraries/llama-downloads/
Request access to gated model on huggingface: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf

Answer 4 · 2023-08-26T20:08:22.000Z

Success 🚀 : https://dbc-464ba720-0425.cloud.databricks.com/?o=3203210999074204#notebook/2579478632523918/command/2579478632523925

Answer 5 · 2023-09-01T18:37:03.000Z

It has been decided we will run HiPPO in databricks w/ MLFlow