A little python script that helps archiving youtube (and other) channels using yt-dlp.
- Generates a simple batch file that you can use to get everything.
- Skips downloading all video metadata, unlike vanilla yt-dlp.
- Keeps track the videos you have without needing to keep the videos accessible, and only dowloads missing stuff.
- Numbers each video, also remembers old numbering (meaning: deleted videos will not screw it up).
- Backs up it's own data before an update.
First make sure that you have python 3 installed.
Install yt-dlp:
sudo pip install yt-dlp
create a new virtual environment in the current folder:
python -m venv venv
This will create a new venv folder.
Activate it:
POSIX:
bash/zsh $ source venv/bin/activate
fish $ source venv/bin/activate.fish
csh/tcsh $ source venv/bin/activate.csh
PowerShell $ venv/bin/Activate.ps1
bash (alternate) $ . ./venv/bin/activate
Windows:
cmd.exe C:\> venv\Scripts\activate.bat
PowerShell PS C:\> venv\Scripts\Activate.ps1
Install the newest yt-dlp (Recommended, as sites change a lot.):
pip install yt-dlp
Or install the last tested version (In case the newest doesn't work):
pip install -r requirements.txt
Here are the full python virtualenv docs if needed.
Set up your config file:
cp config.txt.example config.txt
Add all channels that you want to grab to your new config.txt.
Run the script:
python ytdl_link_gen.py
If you run into any issue you can safely run the script again, it will continue where it left off.
Once this is finished you can (and should, if you want to update videos again) delete the new temp directory which got created. It contains channel data,
so if an error occurs it won't have to re-download channel lists again.
You should have a dl.sh now. Containing yt-dlp commands to all videos since the previous run.
Run it single threaded:
chmod +x dl.sh
./dl.sh
Or for example using parallel:
parallel -j4 -a dl.sh --progress
If you get an error conplaining about a null object being unindexable check the .json for that channel in the temp directory. if it only has a null in in delete it. That should solve the issue.