Loading our own Data
Opened this issue · 2 comments
I'd like to know the specific location within the project files where we can add and upload our own documents or links to be used as memory without having to prompt the app to research them first. For instance, if I want to include all the recent OpenAI documentation as memory, how and where should I do this? Any guidance would be greatly appreciated!
when the vector db is initialized, 2 hardcoded memories are added using addTestData
Line 89 in 273dd45
I'd make it read a folder with the documentation files or just scrape the data from the website, then have the text split/embedded using Langchaing like outlined in here https://github.com/PineappleExpress808/lex-gpt/blob/main/scripts/get_data.ipynb (probably mckaywrigley has already done something similar in his other repos, I recommend you check them out)
when the vector db is initialized, 2 hardcoded memories are added using
addTestDataLine 89 in 273dd45
I'd make it read a folder with the documentation files or just scrape the data from the website, then have the text split/embedded using Langchaing like outlined in here https://github.com/PineappleExpress808/lex-gpt/blob/main/scripts/get_data.ipynb (probably mckaywrigley has already done something similar in his other repos, I recommend you check them out)
@r-fumachi can you make a tuto ? or a quick video to show off this exemple ?