Deploying Falcon to SageMaker TGI DLC after QLoRA fine-tuning

[austinmw]

Hi,

I was able to deploy the base Falcon-40B model to SageMaker using the TGI DLC by following this blog post

I also recently fine-tuned the Falcon-40B model with QLoRA on SageMaker, and obtained the following files:

model/checkpoint-1000/adapter_model/adapter_model.bin
model/checkpoint-1000/adapter_model/adapter_config.json

Now I'm wondering, how do I deploy the model with these adapter weights to TGI DLC on SageMaker?