Info: This repo is set up to use models hosted and accessible via https://build.nvidia.com/
These models are already available and you can use them by creating yourself API keys through the platform. The project is set up so that you can change these models to NIM deployments by setting the
model
name andapi_url
in theNvidiaGenerator
,NvidiaDocumentEmbedder
andNvidiaTextEmbedder
components.👩🏻🍳 We also provide a notebook on Haystack Cookbooks that provide the same code and setup, only expecting self-hosted NIMs
pip install hayhooks
- Create a
.env
file and addNVIDIA_API_KEY
(if you're using hosted models via https://build.nvidia.com/ then add an extraNVIDIA_EMBEDDINGS_KEY
for embedding models) docker-compose up
hayhooks deploy rag.yaml
indexing.py
: This script preproecesses, embeds and writes ChipNemo.pdf into a Qdrant databaserag.py
: This scripts runs a RAG pipeline with a NIM LLM and retrieval model.Dockerfile
: This is used by the docker-compose file to install dependenciesdocker-compose.yml
: This is the docker compose file we use to spin up a container for hayhooks (Haystack pipeline deployment) and Qdrantrag.yaml
: This is the serialized RAG pipeline which is the same asrag.py
in YAML. We use this to deploy our pipeline with hayhooks- : This notebook shows you how you can set up your components to use self-hosted NIMs.