experiment with langchain and pinecone
ingest a PDF langchain breaks it up into documents openai changes these into embeddings - literally a list of numbers. a giant vector in 1500-dimensional space pinecone stores these embeddings externally
openai turns a question into an embedding; pinecone will return the embeddings most similar to that query openai will take those supplied embeddings and return an answer
apt-get update && apt-get -y install pybind11-dev CC=clang CXX=clang++ ARCHFLAGS="-arch x86_64" python -m pip install 'git+https://github.com/facebookresearch/detectron2.git' --user ???
remember, openai embeddings have 1536 dimensions
https://www.youtube.com/watch?v=h0DHDp1FbmQ&ab_channel=DataIndependent
#notes for refactor:
- Format Tara's docs and upload them all to PineCone
- Create a QA bot from those
- Use Just call API to ask questions from the QA bot
- ensure it's remembering the history
- store all conversations in archive