Library to support common tasks in retrieval augmented generation (RAG).
This library is in a very early stage and all the documentation is AI generated.
You find a brief tutorial and the documentation under br-data.github.io/rag-tools-library.
- Add Google Bison to available LLMs
- Add an offline database alternative
- FAISS and SQLite
- Allow users to register their own LLMs
- Allow users to register their own Embedding models
- Support Semantic Scholar endpoint to generate embeddings for scientific papers.
- Support chat functionality; e.g. let the user give feedback on the result to the LLM.
Run the build_and_deploy.sh script in the root folder. Once prompted for the username, pass __token__ and the pypi API
token you've received. If you don't have an API token and feel like you should, feel free to contact the maintainers.
Marco Lehner