Feature Request: Add callback to read the output of process whether it is progress or not
Closed this issue · 5 comments
At the moment, when starting a process via YoutubeDL.getInstance().execute()
, you can specify a method as a callback
parameter that will receive the video download progress.
This is useful, but I would like the library to provide the ability to also get the current output from a process.
Example:
I send this request to get brief information about each video on the channel:
val request = YoutubeDLRequest("https://www.youtube.com/@MrBeast/videos")
request.addOption("--skip-download")
request.addOption("--print-json")
request.addOption("--flat-playlist")
val result = YoutubeDL.getInstance().execute(request, "test", callback)
But at the same time callback
does not receive a single call, because in fact, the download does not occur and yt-dlp
does not output a line that could be read by the library as download progress.
Anyway, for each video, yt-dlp
sends a separate json
to stdout
, which the library reads in line 34 of the StreamProcessExtractor
file:
val line = currentLine.toString()
As a result, instead of sequentially getting information about each video from the process, I have to wait until it is completely loaded and take the final result of yt-dlp
from the result
variable in my code.
Current solution:
-
Wait until the process is fully loaded and completed. For channels with 1000+ videos, this takes a lot of time.
-
Run several processes in parallel at once and receive videos from the channel in portions, for example, at least 5 pieces, through the arguments
--playlist-start $chunkSize
and--playlist-end ${chunkSize+5}
. This wastes a lot of hardware resources.
Ideal solution:
Add another argument to execute
to specify the method that will receive the process output.
What do you mean by sequentially getting information about each video from the process
? I believe that the yt-dlp process does only emit one output for your use case, and that's exact how it works.
Try the command
yt-dlp --skip-download --print-json --flat-playlist https://www.youtube.com/@MrBeast/videos
Or this one that takes longer to complete
yt-dlp --get-title https://www.youtube.com/watch?v=tWYsfOSY9vY https://www.youtube.com/watch?v=KOEfDvr4DcQ https://www.youtube.com/watch?v=krsBRQbOPQ4
And programmatically read the output from the process, for example through a program in Python or in any other language.
Or set a breakpoint on line 34
of the file StreamProcessExtractor.kt
and see how many times the debugger stops at this line and what will be in the line
variable
For a quick walkaround, i think we can just remove this if
statement to expose every line of output to the callback
You mean I should download the source code and do it myself?
You can use the snapshot build from the master branch for the walkaround. But a more robust solution might need modifications to API