simonw/llm

Plugin for HF serverless inference

hugobowne opened this issue · 0 comments

@simonw we just discussed adding support for HF serverless inference: https://huggingface.co/docs/api-inference/en/index

it would be super cool as many fun models have this option:

Screenshot 2024-06-12 at 11 55 16 AM

I don't have experience building such plugins but would be excited to learn!

iirc i could use this to try? https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html