aws/sagemaker-huggingface-inference-toolkit

Sagemaker endpoint inference Fails when following a tutorial

IoannisDem opened this issue · 0 comments

Hello,
I have been trying to use a Llava endpoint in sagemaker when following this tutorial (https://github.com/aws-samples/multimodal-rag-on-slide-decks/blob/main/Blog1-TitanEmbeddings-LVM/notebooks/0_deploy_llava.ipynb).
I havent used the Cloudformation, I am just interested in deploying the model itself.
The endpoint is deployed however, when I try to invoke it i get the following error:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"code": 400,
"type": "InternalServerException",
"message": "GET was unable to find an engine to execute this computation"
}

Not quite sure what to do and what the exact issue is.