aws/sagemaker-huggingface-inference-toolkit

get_pipeline function passes Path object rather than PretrainedTokenizer

jpang32 opened this issue · 0 comments

When using certain pipeline tasks (e.g., "image-to-text") the get_pipeline function passes in model_dir as the tokenizer ink wargs. However, the Pipeline class expects a PretrainedTokenizer object instead. Because model_dir is being passed, when the tokenizer is decoded in the subclass it gives the error "message": "str object has no attribute decode".

This should be changed so that this function is using the proper tokenizer for the task.