StampyAI/alignment-research-dataset

Check pinecone for deleted sources

Closed this issue · 0 comments

Pinecone needs to periodically run through all the entries to make sure they are still valid documents in MySQL. If documents are removed from MySQL and there's an embedding in pinecone, those chunks need to be removed. Do we need to periodically re-sync metadata in case they get updated?