aws/sagemaker-huggingface-inference-toolkit

Serverless Inference

Closed this issue · 3 comments

navidre commented

Hello, and thank you for the open-source code. Does this code support serverless inference on AWS SageMaker as described here?

philschmid commented

Yes, it does.
I wrote a blog post on how to easily use it with cdk: https://www.philschmid.de/serverless-transformers-sagemaker-huggingface
And Julien Simon also created a video on it: https://www.youtube.com/watch?v=cUhDLoBH80o&t=13s&ab_channel=JulienSimon

I hope this helps you getting started.

  • 👍1
navidre commented

Awesome, thank you!

It would be nice to have a reference to them in README or documentations if possible

philschmid commented

We are going to add Documentation & more examples when the feature is out of preview

  • 👍1