jenslys/autovod

[Bug] autovod doesn't support Streamlink's twitch-api-header argument

Closed this issue · 9 comments

BenjiThatFoxGuy commented

Describe the bug
autovod doesn't work properly with twitch-api-header added to the Streamlink arguments in the streamer config

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'streamer-name.config'
  2. Include --twitch-api-header "Authorization=OAuth <twitch-api-header-without-the-<>-brackets>" in the STREAMLINK_FLAGS section of the streamer config
  3. Start autovod
  4. The streamer goes live
  5. Autovod doesn't work, logs reveal that Streamlink complains about invalid arguments.

Seems like autovod isn't passing something properly. Maybe it's not parsing the quotes required for that Streamlink arg (--twitch-api-header) properly.

Expected behavior
Autovod works as usual, and that argument is passed to Streamlink. Which results in ads not being passed through if the api header comes from a subscriber/broadcaster account.
Therefore no inset "Ads in progress" from twitch (or pause in the data stream when using --twitch-disable-ads) so long the api header hasn't expired yet.

Desktop (please complete the following information):

  • OS: [Ubuntu Server 22, headless.]

Additional context
Autovod is being started by pm2.

jenslys commented

Can you test doing something like this:
STREAMLINK_FLAGS="--twitch-api-header 'Authorization=YOUR_AUTH_TOKEN'"

Your original code seems to pass OAuth as the value

BenjiThatFoxGuy commented

that code worked locally in a terminal so I assumed it'd work, that's what I found on how to use it somewhere when I Googled.
I'll see and report back.

BenjiThatFoxGuy commented

streamlink was broken by a recent twitch change so I cannot test. I'll come back to it when it's fixed.

Zfalcor commented

The end output was using AutoVOD for a successful upload using just the auth code without OAuth and space in front, it seems it still waits for prerolls using an auth code from an account with Twitch Turbo. This could be due to OAuth and space being required as part of the token or it could be misinterpreting the segments as ad segments. The variable currently will be split at the space even with small quotes around the variable which causes the auth token itself to be taken as a new unrecognized parameter. According to the streamlink documentation the header variables should be as follows:

twitch-api-header=Authorization=OAuth abcdefghijklmnopqrstuvwxyz0123

https://streamlink.github.io/cli/plugins/twitch.html#authentication

Currently used STREAMLINK_FLAGS in bottom Output:
STREAMLINK_FLAGS="--twitch-disable-hosting --twitch-disable-ads --twitch-disable-reruns --twitch-api-header=Authorization=abcdefghijklmnopqrstuvwxyz0123"

Required STREAMLINK_FLAGS that break at the space after OAuth:
STREAMLINK_FLAGS="--twitch-disable-hosting --twitch-disable-ads --twitch-disable-reruns --twitch-api-header=Authorization=OAuth abcdefghijklmnopqrstuvwxyz0123"

Output from Streamlink (AUTHCODE is a placeholder for real code, as is streamName ) :

[cli][info] streamlink is running as root! Be careful!
[cli][debug] OS:         Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
[cli][debug] Python:     3.10.6
[cli][debug] Streamlink: 5.5.1
[cli][debug] Dependencies:
[cli][debug]  certifi: 2023.5.7
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.9.3
[cli][debug]  pycountry: 22.3.5
[cli][debug]  pycryptodome: 3.18.0
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.31.0
[cli][debug]  urllib3: 2.0.3
[cli][debug]  websocket-client: 1.6.1
[cli][debug]  importlib-metadata: 4.6.4
[cli][debug] Arguments:
[cli][debug]  url=twitch.tv/streamName
[cli][debug]  stream=['best']
[cli][debug]  --loglevel=debug
[cli][debug]  --stdout=True
[cli][debug]  --hls-duration=42900
[cli][debug]  --twitch-disable-ads=True
[cli][debug]  --twitch-disable-hosting=True
[cli][debug]  --twitch-disable-reruns=True
[cli][debug]  --twitch-api-header=[('Authorization', 'AUTHCODE')]
[cli][info] Found matching plugin twitch for URL twitch.tv/streamName
[plugins.twitch][debug] Getting live HLS streams for streamName
[plugins.twitch][debug] {'adblock': False, 'geoblock_reason': '', 'hide_ads': False, 'server_ads': True, 'show_ads': True}
[utils.l10n][debug] Language code: en_US
[cli][info] Available streams: audio_only, 160p (worst), 360p, 480p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)
[plugins.twitch][info] Will skip ad segments
[stream.hls][debug] Reloading playlist
[cli][debug] Pre-buffering 8192 bytes
[plugins.twitch][info] Waiting for pre-roll ads to finish, be patient
[stream.hls][debug] First Sequence: 0; Last Sequence: 2
[stream.hls][debug] Start offset: 0; Duration: 42900; Start Sequence: 0; End Sequence: None
[stream.hls][debug] Adding segment 0 to queue
[stream.hls][debug] Adding segment 1 to queue
[stream.hls][debug] Adding segment 2 to queue
[stream.hls][debug] Discarding segment 0
[stream.hls][info] Filtering out segments and pausing stream output
[stream.hls][debug] Discarding segment 1
[stream.hls][debug] Discarding segment 2
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 3 to queue
[stream.hls][debug] Discarding segment 3
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 4 to queue
[stream.hls][debug] Adding segment 5 to queue
[stream.hls][debug] Adding segment 6 to queue
[stream.hls][debug] Adding segment 7 to queue
[stream.hls][debug] Discarding segment 4
[stream.hls][debug] Discarding segment 5
[stream.hls][debug] Discarding segment 6
[stream.hls][debug] Discarding segment 7
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 8 to queue
[stream.hls][debug] Writing segment 8 to output
[stream.hls][debug] Segment 8 complete
[cli][debug] Writing stream to output
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.
[stream.hls][info] Resuming stream output
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 9 to queue
[stream.hls][debug] Adding segment 10 to queue
[stream.hls][debug] Adding segment 11 to queue
[stream.hls][debug] Adding segment 12 to queue
[stream.hls][debug] Writing segment 9 to output
[stream.hls][debug] Segment 9 complete
[stream.hls][debug] Writing segment 10 to output
[stream.hls][debug] Segment 10 complete
[stream.hls][debug] Writing segment 11 to output
[stream.hls][debug] Segment 11 complete
[stream.hls][debug] Writing segment 12 to output
[stream.hls][debug] Segment 12 complete
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 13 to queue
[stream.hls][debug] Writing segment 13 to output
[stream.hls][debug] Segment 13 complete
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 14 to queue
[stream.hls][debug] Adding segment 15 to queue
[stream.hls][debug] Adding segment 16 to queue
[stream.hls][debug] Adding segment 17 to queue
[stream.hls][debug] Writing segment 14 to output
[stream.hls][debug] Segment 14 complete
[stream.hls][debug] Writing segment 15 to output
[stream.hls][debug] Segment 15 complete
[stream.hls][debug] Writing segment 16 to output
[stream.hls][debug] Segment 16 complete
[stream.hls][debug] Writing segment 17 to output
[stream.hls][debug] Segment 17 complete
jenslys commented

The end output was using AutoVOD for a successful upload using just the auth code without OAuth and space in front, it seems it still waits for prerolls using an auth code from an account with Twitch Turbo. This could be due to OAuth and space being required as part of the token or it could be misinterpreting the segments as ad segments. The variable currently will be split at the space even with small quotes around the variable which causes the auth token itself to be taken as a new unrecognized parameter. According to the streamlink documentation the header variables should be as follows:

twitch-api-header=Authorization=OAuth abcdefghijklmnopqrstuvwxyz0123

streamlink.github.io/cli/plugins/twitch.html#authentication

Currently used STREAMLINK_FLAGS in bottom Output: STREAMLINK_FLAGS="--twitch-disable-hosting --twitch-disable-ads --twitch-disable-reruns --twitch-api-header=Authorization=abcdefghijklmnopqrstuvwxyz0123"

Required STREAMLINK_FLAGS that break at the space after OAuth: STREAMLINK_FLAGS="--twitch-disable-hosting --twitch-disable-ads --twitch-disable-reruns --twitch-api-header=Authorization=OAuth abcdefghijklmnopqrstuvwxyz0123"

Output from Streamlink (AUTHCODE is a placeholder for real code, as is streamName ) :

[cli][info] streamlink is running as root! Be careful!
[cli][debug] OS:         Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
[cli][debug] Python:     3.10.6
[cli][debug] Streamlink: 5.5.1
[cli][debug] Dependencies:
[cli][debug]  certifi: 2023.5.7
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.9.3
[cli][debug]  pycountry: 22.3.5
[cli][debug]  pycryptodome: 3.18.0
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.31.0
[cli][debug]  urllib3: 2.0.3
[cli][debug]  websocket-client: 1.6.1
[cli][debug]  importlib-metadata: 4.6.4
[cli][debug] Arguments:
[cli][debug]  url=twitch.tv/streamName
[cli][debug]  stream=['best']
[cli][debug]  --loglevel=debug
[cli][debug]  --stdout=True
[cli][debug]  --hls-duration=42900
[cli][debug]  --twitch-disable-ads=True
[cli][debug]  --twitch-disable-hosting=True
[cli][debug]  --twitch-disable-reruns=True
[cli][debug]  --twitch-api-header=[('Authorization', 'AUTHCODE')]
[cli][info] Found matching plugin twitch for URL twitch.tv/streamName
[plugins.twitch][debug] Getting live HLS streams for streamName
[plugins.twitch][debug] {'adblock': False, 'geoblock_reason': '', 'hide_ads': False, 'server_ads': True, 'show_ads': True}
[utils.l10n][debug] Language code: en_US
[cli][info] Available streams: audio_only, 160p (worst), 360p, 480p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)
[plugins.twitch][info] Will skip ad segments
[stream.hls][debug] Reloading playlist
[cli][debug] Pre-buffering 8192 bytes
[plugins.twitch][info] Waiting for pre-roll ads to finish, be patient
[stream.hls][debug] First Sequence: 0; Last Sequence: 2
[stream.hls][debug] Start offset: 0; Duration: 42900; Start Sequence: 0; End Sequence: None
[stream.hls][debug] Adding segment 0 to queue
[stream.hls][debug] Adding segment 1 to queue
[stream.hls][debug] Adding segment 2 to queue
[stream.hls][debug] Discarding segment 0
[stream.hls][info] Filtering out segments and pausing stream output
[stream.hls][debug] Discarding segment 1
[stream.hls][debug] Discarding segment 2
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 3 to queue
[stream.hls][debug] Discarding segment 3
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 4 to queue
[stream.hls][debug] Adding segment 5 to queue
[stream.hls][debug] Adding segment 6 to queue
[stream.hls][debug] Adding segment 7 to queue
[stream.hls][debug] Discarding segment 4
[stream.hls][debug] Discarding segment 5
[stream.hls][debug] Discarding segment 6
[stream.hls][debug] Discarding segment 7
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 8 to queue
[stream.hls][debug] Writing segment 8 to output
[stream.hls][debug] Segment 8 complete
[cli][debug] Writing stream to output
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.
[stream.hls][info] Resuming stream output
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 9 to queue
[stream.hls][debug] Adding segment 10 to queue
[stream.hls][debug] Adding segment 11 to queue
[stream.hls][debug] Adding segment 12 to queue
[stream.hls][debug] Writing segment 9 to output
[stream.hls][debug] Segment 9 complete
[stream.hls][debug] Writing segment 10 to output
[stream.hls][debug] Segment 10 complete
[stream.hls][debug] Writing segment 11 to output
[stream.hls][debug] Segment 11 complete
[stream.hls][debug] Writing segment 12 to output
[stream.hls][debug] Segment 12 complete
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 13 to queue
[stream.hls][debug] Writing segment 13 to output
[stream.hls][debug] Segment 13 complete
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 14 to queue
[stream.hls][debug] Adding segment 15 to queue
[stream.hls][debug] Adding segment 16 to queue
[stream.hls][debug] Adding segment 17 to queue
[stream.hls][debug] Writing segment 14 to output
[stream.hls][debug] Segment 14 complete
[stream.hls][debug] Writing segment 15 to output
[stream.hls][debug] Segment 15 complete
[stream.hls][debug] Writing segment 16 to output
[stream.hls][debug] Segment 16 complete
[stream.hls][debug] Writing segment 17 to output
[stream.hls][debug] Segment 17 complete

Have u tried updating streamlink to v 6.0.0?
I know there has been a lot of changes to the Twitch plugin.
https://github.com/streamlink/streamlink/releases/tag/6.0.0

I can look into this when I get home next week.

Zfalcor commented

I have updated but am still having a lot of discarded segments/ad segments. My guess is the OAuth is required as part of the Authorization token as stated in the Streamlink docs. The current implementation doesn't allow for a space in STREAM_FLAGS as it considers it a new parameter after a space. Thanks for looking into it when you have time.

jenslys commented

@Zfalcor @ddomino007 There is an open PR (#71) that fixes the issue.
I don't have any active subscriptions to anybody, so do you mind testing it out?

Zfalcor commented

The OAuth code is passed successfully and I receive 'show_ads':False instead of show_ads':True so it seems to be working with an account with a sub or Twitch Turbo. Thank you for the update.

[14:34:09.452327][plugins.twitch][debug] {'adblock': False, 'geoblock_reason': '', 'hide_ads': True, 'server_ads': True, 'show_ads': False}

I am trying to test a short upload but have had issues with the cut-off times or manually stopping without interfering with the youtube upload. Usually, they fail to start processing as if the upload stream never ends. Using --twitch-disable-ads filter for streamlink doesn't allow for hls duration filtering according to --hls-duration and ad filtering #5100 which is why I was looking to get the auth code working.

I did not get the Stream uploaded to youtube message since I think I stopped the app too early. I will try again with a 5-minute limit without stopping the app early so I can get a properly processed video to verify the update.

[16:26:28.021399][stream.hls][info] Stopping stream early after 300
[16:26:28.021508][stream.segmented][debug] Closing worker thread
[16:26:28.362793][stream.hls][debug] Writing segment 1582 to output
[16:26:28.364790][stream.hls][debug] Segment 1582 complete
[16:26:28.587600][stream.hls][debug] Writing segment 1583 to output
[16:26:28.588762][stream.hls][debug] Segment 1583 complete
[16:26:28.803957][stream.hls][debug] Writing segment 1584 to output
[16:26:28.804874][stream.hls][debug] Segment 1584 complete
[16:26:28.804987][stream.segmented][debug] Closing writer thread
[16:26:29.534792][cli][info] Stream ended
[16:26:29.534995][cli][info] Closing currently open stream...

With 00:05:00 duration I successfully uploaded a test video, no ads/purple screens were displayed or discarded fragments. The video was uploaded with the correct title/description from meta info and the correct playlist from STREAMLINK_FLAGS array. This should mean testing is complete for update #71.

The end duration was 00:05:01 which explains why a limit of 12:00:00 or 11:59:59 fails to upload. The cut-off is slightly longer due to hls segment length/calculations.

[17:11:50.252152][stream.hls][debug] Adding segment 2942 to queue
[17:11:50.252367][stream.hls][info] Stopping stream early after 300
[17:11:50.252476][stream.segmented][debug] Closing worker thread
[17:11:50.632082][stream.hls][debug] Writing segment 2942 to output
[17:11:50.633190][stream.hls][debug] Segment 2942 complete
[17:11:50.633303][stream.segmented][debug] Closing writer thread
[17:12:11.449072][cli][info] Stream ended
[17:12:11.449297][cli][info] Closing currently open stream...


17:06:50| All required files found
17:12:26| Stream uploaded to youtube
BenjiThatFoxGuy commented

Hey there! Sorry for no replies, I was a bit busy with life and stuff. I had no way to check for ads because I have no turbo and the only sub maintain is through prime - the streamer isn’t live right now though. I could make a random stream on my own channel but I think you can imagine that would be a.. weird way to test if I am to ping everyone :)
Closing this assuming everything is working, please reopen if needed.