/ElasticDocs_GPT

Combining the search power of Elasticsearch with the Question Answering power of GPT

Primary LanguageJupyter NotebookGNU General Public License v3.0GPL-3.0

ElasticDocs_GPT

Combining the search power of Elasticsearch with the Question Answering power of GPT

Blog - ChatGPT and Elasticsearch: OpenAI meets private data

diagram

  1. Python interface accepts user questions
  • Generate a hybrid search request for Elasticsearch
  • BM25 match on the title field
  • kNN search on the title-vector field
  • Boost kNN search results to align scores
  • Set size=1 to return only the top scored document
  1. Search request is sent to Elasticsearch
  2. Documentation body and original url are returned to python
  3. API call is made to OpenAI ChatCompletion
  • Prompt: "answer this question using only this document <body_content from top search result>"
  1. Generated response is returned to python
  2. Python adds on original documentation source url to generated response and prints it to the screen for the user

Examples

autoscale

apm

inference

pii