These scripts aid in removing/preventing duplicates with youtube-dl. See YTDL Issue #23502 for more information.
There are two scripts here:
- One to auto-generate archive file
- Another to remove duplicates
You must have Node.js downloaded & installed to run these scripts: https://nodejs.org/en/
Download/clone these scripts to your home directory (or wherever you like)
cd ~/
git clone https://github.com/DrewImm/ytdl-scripts
Change directory to the directory of your video files
node ~/ytdl-scripts/archive-writer.js
node ~/ytdl-scripts/dupe-remover.js