taalbrecht/JournalVis

Add function to eliminate duplicate documents

taalbrecht opened this issue · 0 comments

taalbrecht commented

Add a check for duplicate documents (identical contents and/or checksum) when ingesting from a local folder.