SageMaker fails because Conversation object is not found

[adrihercer]

We are hosting a model in SageMaker and today we observed the following error in our logs when the model was being relaunched in the instance:

ImportError: cannot import name 'Conversation' from 'transformers.pipelines' (/opt/conda/lib/python3.9/site-packages/transformers/pipelines/__init__.py)

I found someone reported a similar issue in https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556. When digging into the changes in the transformers dependency I found this change regarding the Conversation object: huggingface/transformers#31165

Based in our logs, the last time the model was successfully relaunched in our SageMaker infrastructure (2 days ago), the version of the package that was downloaded was transformers-4.41.2-py3-none-any.whl, but when the error started to be observed, the downloaded version was transformers-4.42.1-py3-none-any.whl.

According to the pull request mentioned above, the change is effective as per version 4.42, which was released 18 hours ago at the moment of writing this issue.

I think a change is needed in src/sagemaker_huggingface_inference_toolkit/transformers_utils.py to reflect the new structure of the code in the transformers dependency or pin the dependency version to version 4.41.2 to prevent the issue in the future.

In our case, we are updating our requirements.txt file to pin the version 4.41.2, but other users might not be aware of what is happening, therefore they are not aware of this fix.