API Key Authentication for Michaelfeil Infinity
Closed this issue · 8 comments
Model description
Could you please provide guidance on how to enable API_KEY authentication? Alternatively, is there a plan to implement API_KEY authentication similar to OpenAI's approach?
Open source status
- The model implementation is available on transformers
- The model weights are available on huggingface-hub
- I verified that the model is currently not running in infinity
Provide useful links for the implementation
No response
It's just a fastapi server, implement it as usual like you do on FastAPI.
I added an example on how to use infinity with runpod.io
note, that API_Keys are typically not set, they are generated. Also they are not verified by the application itself, but by e.g. AWS API Gateway. I understand ypur interest, but i think such a feature is easy to implement yourself or better be added with another service
@monotykamary has implemented it. Changed my mind: If the contribution is easy enough, comes with a unit test that covers every LOC, and defaults to no api key as default behauviour, would be accepted
Oh, I just did a dirty implementation to quickly spin up a quick embedding server with auth on Modal: https://github.com/dwarvesf/llm-hosting/blob/main/infinity_snowflake_arctic_embed_l_335m.py
Looks not dirty at all @monotykamary! Awesome
@monotykamary @semoal @AjayKarma05 Added auth, you can set a Bearer token via INFINITY_API_KEY=mykey123 or --api-key mykey123
How to set the API_KEY in the Langchain Infinity Embedding?
@Jimmy-Newtron PRs to langchain are welcome. I think this feature is missing there.