List of proposed changes to MacYTDL - as at 31/7/24
section83 opened this issue · 32 comments
This is a list of features that might be added in the future. It is not in priority order. If you would like one of these or any other feature added to MacYTDL, please post an issue or discussion, or send an email (address is in the "About" dialog.
- Look at whether “--merge-output-format” is a useful option to add.
- Only if merging video and audio files.
- Add settings for Adobe Pass credentials, video password and .netrc credentials.
- Redesign Help as a Wiki published on GitHub.
- Add settings for “--autonumber-start” and “--output-na-placeholder”.
- Redesign save/restore settings.
- Consider whether/how to initiate parallel downloads when user pastes multiple URLs.
- Currently downloads serially by one YTDL request. Could go parallel – could be a setting.
- Enable user to save one or more set of login credentials – could use KeyChain.
- Provide UI for config file functionality – could have more than one switchable config.
- Provide a macOS "Shortcut" to replace/supplement the current "Send-URL-to_MacYTDL" Service.
- Option on installation to add shortcut to Dock.
- Involves killing the Dock process (which usually restarts) so that change to a plist file is implemented.
- Organising downloads by uploader/downloader
- Is this of any benefit ?
- Facility to get cookies from current browser.
- Add option to control downloads according to date ?
- Option to schedule downloads for a later time ?
- Add "Archives" functionality.
- Facility to search log files.
- Enable the Monitor dialog to be hidden/minimised and non-modal – very difficult using AppleScript.
- Enable user to position the Monitor dialog – tricky as each dialog is a separate process.
- Might be possible to keep separate download processes but one dialog.
- Option to turn off notifications when downloads started/finished.
- Add option to choose playlist items for download – currently always downloads entire list.
- Add option to choose downloads from batch file – currently always downloads entire batch.
- Only remove items from batch list which have been successfully downloaded.
- Enable download of duplicates – perhaps with an index to differentiate files.
- Add option to store YTDL settings with each URL in a batch – would enable different settings for each download.
- Could have one set of settings for each batch list.
- Facility to create more than one batch list and choose which to download.
- Add facility to recode codecs inside a container.
- Provide a file count for playlists, batches and multiple downloads with progress e.g. "2 of 6 files completed".
- Option to add shortcut to Dock.
- Add an abort on error setting – on/off.
- Add support for Homebrew and other installations of FFmpeg and YT-DLP/YTDL.
about ''24. Add option to delete URLs from batch file after download.''
as i understand, currently it deletes all of them if all the batch succeeded,
and doesn't remove anything at all, if even 1 of them failed / interrupted due to internet disconnect etc'....
i think the best default behavement is that it removes only the links of the successful downloads, while leaving these that failed / had issues / weren't downloaded.... if its not too troublesome to implement?
also about ''27. Add function to reset all settings to default.''
if you add something like that, then please..
make sure if you click this option, it gives a warning window with ''are you sure?'' and that 'cancel' is the default selected option for that window....
as such an option scares me XD especially when i can't export a backup of the settings i guess? lol
actually does backing up and restoring 'MacYTDL.plist' serve as a settings backup?
Zxzzz8, many thanks for all that feedback. It really does help.
i think the best default behavement is that it removes only the links of the successful downloads, while leaving these that failed / had issues / weren't downloaded.
Yes, that would be best. It will take a fair amount of code crunching to follow each download. It will be difficult as the output from YT/YT-DLP is different depending on how much FFmpeg is used. The change would work in combination with idea 25 perhaps.
It ties in with idea 3 which I think is a priority.
about ''27. Add function to reset all settings to default.'' if you add something like that, then please..make sure if you click this option, it gives a warning window with ''are you sure?'' and that 'cancel' is the default selected option for that window.
Can do.
does backing up and restoring 'MacYTDL.plist' serve as a settings backup?
Yes, but, "MacYTDL.plist" is the only settings file currently used. So, you can manually rename it and a new plist file will be created on next startup. You can manage the various plists using their names (e.g. "MacYTDL-youtube.plist") and rename the one you want active to "MacYTDL.plist". I could add a function that does that. Just beware that removing or renaming "MacYTDL.plist" while MacYTDL is running will cause a crash (at present).
Cheers.
actually i was worried that mby im bothering you too much XD
but i guess i assumed i don't need to feel that way with how you respond? ^^
oh and about ''3. Provide for downloads to be paused and continued – possibly by adding "Keep files" option to the "Stop" function.''
can that also include the ability to not break the file when there is an internet disconnect?
i have quite an unstable internet recently in the past few years, and on top of that, its standard where i live to have dynamic ip, so that also breaks a file when that happens....
Not sure what you mean by "break the file". Both youtube-dl and YT-DLP will continue a partial download. I have tested that and it works well. You will possibly see files with ".part" extension. That's a partial download. But the file is possibly not playable until complete.
Cheers.
i see, it gave me that impression, sense when i downloaded a batch, after a disconnect, it did not try to continue the partial file it stopped in the middle, but instead leave it like that and move to the next....
then i guess what i ask is that after a disconnect when using the batch tool, it will not move to the next file after a disconnect [or finish the process if its the last file], but instead retry to continue the file it stopped first....
I have very little control over how youtube-dl/YT-DLP handle batches. I can't interrupt the Python process and force it to go back and try again. Also, there can be many causes of errors which youtube-dl/YT-DLP don't trap.
Nonetheless, I can monitor a batch download and when finished get MacYTDL to go back, find any errors and try again. However, it's tricky to test as my Internet connection does not disconnect – maybe I can test it by unplugging my Ethernet cable during a batch download !
There is an option to "abort-on-error" which might be what you're after. I'll add that as an option in "Settings". MacYTDL does already report that there were error(s) in a download. Abort-on-error will prevent the youtube-dl/YT-DLP process from continuing when there are problems such as flakey Internet connections.
section83--''Nonetheless, I can monitor a batch download and when finished get MacYTDL to go back, find any errors and try again....''
if you can do that^, then that might be the best solution for me, if it can check the broken files and finish downloading them, after finishing the main batch....
btw if you do that solution, do you think there is a chance that it will lose a few seconds part of the video where it brute-stopped due to internet loss?
btw if you do that solution, do you think there is a chance that it will lose a few seconds part of the video where it brute-stopped due to internet loss?
youtube-dl/YT-DLP continue downloading from the point in the file at which it stopped. It doesn't matter how long the time is between downloading being stopped and being continued as long as the "part" file is available.
so the brute-stop won't corrupt some data if i understand correctly?
re 17 Enable finer choices on codecs ?
Is there way to output YT's webm as something that works with quicklook?
As a first step, MacYTDL can remux from webm to mp4, mkv, ogg, avi and flv. But, I guess you found that while the video is playable in Quick Look, the audio is not. That's because the remux only copies the codecs across to a new container. YouTube webm files usually have audio using the Opus codec which Quick Look can't decode. Idea 17 would provide a way to re-encode video and/or audio into chosen codecs (which would be compatible with Quick Look).
Note that re-encoding takes a lot longer – on my (very slow) iMac re-encoding the video takes maybe 50 times the time. One video I tested took well over an hour.
Anyway, it's a useful option so, I'll push 17 up the priority list.
Thanks.
Yes, the remuxed files are playable in many players (e.g. VLC). But, for some reason, Apple has not included decoders for some codecs in their macOS frameworks/libraries.
I really appreciate the work you’ve done on this. Can’t rely on any video remaining on the net, so it’s really good to be able to archive stuff locally.
My pleasure – it has been a rewarding retirement occupation. Using a downloader can make watching videos much easier. I've had a lot of trouble trying to watch catch-up TV shows in a web browser. It's often flakey and controlling is painful with a keyboard or even a universal remote.
Any plans to compile as native Apple Silicon (M1)?
Any plans to compile as native Apple Silicon (M1)?
On an M1 Mac, MacYTDL currently runs under Rosetta2. I have been able to borrow an iMac running macOS Monterey and Script Debugger 8. So, v1.21 of MacYTDL will be in universal format.
Eventually, I will acquire an M1 Mac and will be able to compile an M1 specific version as well. I doubt there will be a significant speed difference.
Cheers.
Any plans to compile as native Apple Silicon (M1)?
On an M1 Mac, MacYTDL currently runs under Rosetta2. I have been able to borrow an iMac running macOS Monterey and Script Debugger 8. So, v1.21 of MacYTDL will be in universal format.
Eventually, I will acquire an M1 Mac and will be able to compile an M1 specific version as well. I doubt there will be a significant speed difference.
Cheers.
Nice!! From my experience, I can affirm the performance is significant (and resources consumption also). Talking exclusively about AdGuard. For a long was running under Rosetta2... take some time to load and open and was consuming ~150MB RAM. After update to a native one, the app open almost instantly and consuming is around 70-80MB RAM. It's a good combination RAM and Speed. I can bet ~70% faster without Rosetta2.
I don't know how to do it, if so, I could fork the project and create a new release for Apple Silicon and give you to release to the project.
I've created an arm64 version of MacYTDL. It is available here:
https://github.com/section83/MacYTDL/blob/master/Components/MacYTDL-arm64-test.zip
I have code signed and notarised that copy. Please beware, I took a pre-release copy of MacYTDL v1.21. So, it might have bugs and look ugly in places.
As far I understand, the only difference between x86_64 and arm64 versions is a small MacOS executable which is the AppleScript runner. So, the arm64 version should not be much faster than the Universal version – macOS will run the arm64 code inside a Universal arch executable.
I've converted that executable to arm64. I don't have an M1 Mac so, please post here how the results of your testing.
If the version I created works, you can create your own arm64 version any time using the same procedure as I used:
-
download the unsigned copy of MacYTDL;
-
open Terminal;
-
run this command:
lipo [path to]/MacYTDL.app/Contents/MacOS/applet -thin arm64 -output [path to]/MacYTDL.app/Contents/MacOS/applet2
where "[path to]" is where your downloaded copy is saved.
- right-click MacYTDL;
- navigate to the "MacOS" folder;
- delete "applet";
- rename "applet2" to "applet";
- if you are a member of the Apple Developer Program you can code sign and notarise the applet.
I've created an arm64 version of MacYTDL. It is available here:
https://github.com/section83/MacYTDL/blob/master/Components/MacYTDL-arm64-test.zip
I have code signed and notarised that copy. Please beware, I took a pre-release copy of MacYTDL v1.21. So, it might have bugs and look ugly in places.
As far I understand, the only difference between x86_64 and arm64 versions is a small MacOS executable which is the AppleScript runner. So, the arm64 version should not be much faster than the Universal version – macOS will run the arm64 code inside a Universal arch executable.
I've converted that executable to arm64. I don't have an M1 Mac so, please post here how the results of your testing.
If the version I created works, you can create your own arm64 version any time using the same procedure as I used:
- download the unsigned copy of MacYTDL;
- open Terminal;
- run this command:
lipo [path to]/MacYTDL.app/Contents/MacOS/applet -thin arm64 -output [path to]/MacYTDL.app/Contents/MacOS/applet2where "[path to]" is where your downloaded copy is saved.
- right-click MacYTDL;
- navigate to the "MacOS" folder;
- delete "applet";
- rename "applet2" to "applet";
- if you are a member of the Apple Developer Program you can code sign and notarise the applet.
Nice to know you found the difference. About the performance (Universal X arm64) I think won't be significant but what I see it's Universal takes more storage than a dedicated app, maybe because the app stores more code to support Intel and arm?
I've created an arm64 version of MacYTDL. It is available here:
https://github.com/section83/MacYTDL/blob/master/Components/MacYTDL-arm64-test.zip
I have code signed and notarised that copy. Please beware, I took a pre-release copy of MacYTDL v1.21. So, it might have bugs and look ugly in places.
As far I understand, the only difference between x86_64 and arm64 versions is a small MacOS executable which is the AppleScript runner. So, the arm64 version should not be much faster than the Universal version – macOS will run the arm64 code inside a Universal arch executable.
I've converted that executable to arm64. I don't have an M1 Mac so, please post here how the results of your testing.
If the version I created works, you can create your own arm64 version any time using the same procedure as I used:
- download the unsigned copy of MacYTDL;
- open Terminal;
- run this command:
lipo [path to]/MacYTDL.app/Contents/MacOS/applet -thin arm64 -output [path to]/MacYTDL.app/Contents/MacOS/applet2where "[path to]" is where your downloaded copy is saved.
- right-click MacYTDL;
- navigate to the "MacOS" folder;
- delete "applet";
- rename "applet2" to "applet";
- if you are a member of the Apple Developer Program you can code sign and notarise the applet.
Nice to know you found the difference. About the performance (Universal X arm64) I think won't be significant but what I see it's Universal takes more storage than a dedicated app, maybe because the app stores more code to support Intel and arm?
About the test of the Apple Silicon version, it seems works fine, no bug noticed but the performance really wasn't improved. From click to show the window took around 15 seconds against 20 seconds from non-apple silicon.
About RAM Consumption
maybe because the app stores more code to support Intel and arm?
Yes, the applet executable is larger because it has both x86_64 and arm64 code.
the performance really wasn't improved
That makes sense. The Universal build includes arm64 code which is what an M1 Mac will run.
It makes sense also that the memory footprint is greater as there is probably some overhead required to separate out and run the arm64 code. The number of threads might be relevant too.
Cheers.
I know you didn't asked but if you like you can use it. I just create a simple new logo (macOS BigSur Style) for MacYTDL. If you liked or need some improvements or more versions in lower resolutions just call me on telegram @gustavosaez .
Have you considered that a user may have ffmpeg, ffprobe, youtube-dl and/or yt-dlp already installed on their system? If they are using homebrew, on Intel it will be in /usr/local/bin (no issue there), but on ARM it is now in /opt/homebrew/bin (a new change in homebrew). This would make it easier to see if it already installed, and also may prevent users from having version conflicts (depending on path variable).
You could also detect if homebrew is installed and give users the option to install through there as they are more likely to keep those versions up to date.
Have you considered that a user may have ffmpeg, ffprobe, youtube-dl and/or yt-dlp already installed on their system?
Good question. I have looked at it but not in depth. In one case, a user had installed the Linux version of YT-DLP which caused errors. Other users have installed various Python versions in various ways which also caused some grief.
I didn't know Homebrew now puts executables in /opt/homebrew/bin on ARM Macs. Why did they change from /usr/local/bin ? Does that mean you have to add the path to /opt/homebrew/bin for YT-DLP to find FFmpeg ?
Does Homebrew install the correct version of YT-DLP ? i.e. "yt-dlp_macos" ?
You could also detect if homebrew is installed and give users the option to install through there as they are more likely to keep those versions up to date.
Yes, probably can do that. Do different versions of Homebrew live in different places ? Given that it's possible to install all the tools manually, i.e. without Homebrew, it might be useful to check other locations.
Currently the version number of the installed YTDL/YT-DLP is stored in MacYTDL.plist. I put it there as getting the current version by running YT-DLP was taking a long time every time the user started MacYTDL or opened Utilities (YT-DLP has to expand its Python runtime). It was a pain. Downside is that MacYTD currently doesn't know about copies of YTDL/YT-DLP installed elsewhere, installed/updated by another user on the same Mac or manually updated. Fixing that is on my todo list.
Thanks.
The the main motivation for the change was to allow the transition from Intel to Apple Silicon.
In #9117, we switched to a new prefix of /opt/homebrew for installations on Apple Silicon. This was written and shipped with heroic speed to help prevent strange issues with bleeding edge users on the first consumer Apple Silicon Macs.
—Misty De Méo - Homebrew Maintainer
It could be possible to move everything back to /usr/local/bin in the future, but there are other reasons for sticking with /opt/homebrew even after the Intel Macs are long gone.
Homebrew is not the only tool that installs things in /usr/local/bin and so the potential for conflicts has always been an issue.
There are security concerns with using /usr/local/bin.
Other package managers have been using /opt/<manager_name> for a while now.
So in the long run this is a positive change, but not without a few growing pains along the way.
Many thanks for all this. It's a great help. I've added Homebrew and other self-installs to my list for changes in v1.24.
In #9117, we switched to a new prefix of /opt/homebrew for installations on Apple Silicon. This was written and shipped with heroic speed to help prevent strange issues with bleeding edge users on the first consumer Apple Silicon Macs.
As a Mac user, I'm grateful for such efforts. I hope that someone bought them a coffee, at least.
onaforeignshore – thank you for raising this. It's also alerted me to an improvement in YT-DLP that I missed. There is now a version of YT-DLP, yt-dlp_macos_legacy, which runs on macOS 10.9 to 10.15. I missed the clue provided in release notes for yt-dlp 2022.06.29. Ah well, better late than never.
I've added /opt/homebrew/bin to the temporary $PATH MacYTDL uses to run YT-DLP/youtube-dl. Are there other locations which should be added ?
UPDATE: It looks like Homebrew installs on Intel Macs is a bit of a pain. For example, after installing on a Mac Mini running OSX 10.13, I found YT-DLP was in "/usr/local/Cellar/yt-dlp/2023.3.4/libexec/bin/yt-dlp". So, the location changes every time YT-DLP is updated. There is an alias in /usr/local/Cellar/yt-dlp/2023.3.4/bin which is no better. I can't find any aliases in a friendly location. Also, the version is a different format to that used by the YT-DLP developers.
Moving to /opt/homebrew/bin was a very good idea.
UPDATE 2: Have decided to only use Homebrew install of YT-DLP if there is no MacYTDL install. Also, MacYTDL will not check/update a Homebrew install.
Hi, just wondering if there's a quality control? I'm trying to download HD from 7Plus. It downloads fine but always downloads the lesser quality one instead. I know it's a little trivial but maybe something to look into?
MacYTDL uses the Python script "YT-DLP" to download videos. YT-DLP defaults to download the best resolution available. That's not easy given the variety of video-only, audio-only and audio+video downloads in different formats that can be available. The YT-DLP developers explain:
The default format selector is bv*+ba/b. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be preferred.
In MacYTDL, the default file format uses the YT-DLP default.
7Plus is like SBS and the ABC – although free-to-air HD is 1080p, they often only provide 720p for streaming or even worse. For example, the resolution 7Plus provide for RFDS streaming is 720p – but, YT-DLP can see those files only if you use the cookies options e.g. --cookies-from-browser. You can supply the cookies options to MacYTDL using "Custom settings".
Cheers.
UPDATE: It looks like Homebrew installs on Intel Macs is a bit of a pain. For example, after installing on a Mac Mini running OSX 10.13, I found YT-DLP was in "/usr/local/Cellar/yt-dlp/2023.3.4/libexec/bin/yt-dlp". So, the location changes every time YT-DLP is updated. There is an alias in /usr/local/Cellar/yt-dlp/2023.3.4/bin which is no better. I can't find any aliases in a friendly location. Also, the version is a different format to that used by the YT-DLP developers.
It seems there was an issue with the link on Intel Mac, but it has been resolved - see this
UPDATE 2: Have decided to only use Homebrew install of YT-DLP if there is no MacYTDL install. Also, MacYTDL will not check/update a Homebrew install.