HF_TASK Enviournment Variable error
316usman opened this issue · 1 comments
I have tried to deploy an embedding model in AWS Sagemaker endpoint using the provided guide that uses inference.py to deploy custom code. the endpoint is created and starts but when I query the end-point I get the following error.
"You need to define one of the following ['audio-classification', 'automatic-speech-recognition', 'feature-extraction', 'text-classification', 'token-classification', 'question-answering', 'table-question-answering', 'fill-mask', 'summarization', 'translation', 'text2text-generation', 'text-generation', 'zero-shot-classification', 'conversational', 'image-classification', 'image-segmentation', 'object-detection'] as env 'HF_TASK’
How do I set the enviournment variable in the note book I am using for deployment or inside the inference.py file.
Also is it necessary to keep ‘model_fn’ and ‘predict_fn’ name the same in the inference.py file
please guide I have been stuck for quite a while now and changing anything in the inference.py files requires me to generate the .tar.gz file again which takes alot of time
@316usman Are you using a custom model_fn function in your inference.py ?
This error message is probably caused by the default load function of the HuggingFace handler_service, see here.
This means that Sagemaker is not able to find a custom model_fn in your inference.py (or can't find your inference.py), thus it's using the default one provided by the HuggingFace base image.
And yes, it's required to keep model_fn, predict_fn, input_fn and output_fn names the same, because this is how the handler_service will retrieve them: see here