- Here use Huggingface dataset (https://huggingface.co/datasets/tweet_eval)
- Dataset is divided into small chunks
- Each chunk of text is handed to a sentence transformer, which generates a dense vector to represent its semantic meaning as a vector embedding.Here i use Hugging face embedding (sentence-transformers/all-mpnet-base-v2)
- The chunks are stored in Elasticsearch with their vector embeddings
- When a question is asked, a vector is created for the question and then Elasticsearch is queried to find the semantically closest chunk to the question.
- A prompt template is composed for a LLM that the retrieved text chunk in as extra contextual knowledge. I use HuggingfacePipeline to run the model. (model_id = 'google/flan-t5-large')
- The LLM serve a relevant answer to the question.
a) locally with Docker b) with Elastic Cloud URL (you have to create a cluster id, host ,username and password).