HoloArchivists/twspace-dl

An exception will be thrown if DynamicURL or MasterURL is specified.

migimigi opened this issue · 2 comments

migimigi commented

Describe the bug
An exception will be thrown if DynamicURL or MasterURL is specified.

To Reproduce
DynamicURL

$ twspace_dl -d 'https://prod-fastly-ap-northeast-1.video.pscp.tv/Transcoding/v1/hls/****/non_transcode/ap-northeast-1/periscope-replay-direct-prod-ap-northeast-1-public/audio-space/playlist_****.m3u8?type=replay'
Traceback (most recent call last):
  File "/opt/homebrew/bin/twspace_dl", line 8, in <module>
    sys.exit(main())
  File "/opt/homebrew/lib/python3.10/site-packages/twspace_dl/__main__.py", line 203, in main
    args.func(args)
  File "/opt/homebrew/lib/python3.10/site-packages/twspace_dl/__main__.py", line 72, in space
    twspace = Twspace.from_space_url(args.input_url)
  File "/opt/homebrew/lib/python3.10/site-packages/twspace_dl/twspace.py", line 169, in from_space_url
    space_id = re.findall(r"(?<=spaces/)\w*", url)[0]
  File "/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/re.py", line 240, in findall
    return _compile(pattern, flags).findall(string)
TypeError: expected string or bytes-like object

MasterURL

$ twspace_dl -f 'https://prod-fastly-ap-northeast-1.video.pscp.tv/Transcoding/v1/hls/****/non_transcode/ap-northeast-1/periscope-replay-direct-prod-ap-northeast-1-public/audio-space/master_playlist.m3u8'
Traceback (most recent call last):
  File "/opt/homebrew/bin/twspace_dl", line 8, in <module>
    sys.exit(main())
  File "/opt/homebrew/lib/python3.10/site-packages/twspace_dl/__main__.py", line 203, in main
    args.func(args)
  File "/opt/homebrew/lib/python3.10/site-packages/twspace_dl/__main__.py", line 72, in space
    twspace = Twspace.from_space_url(args.input_url)
  File "/opt/homebrew/lib/python3.10/site-packages/twspace_dl/twspace.py", line 169, in from_space_url
    space_id = re.findall(r"(?<=spaces/)\w*", url)[0]
  File "/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/re.py", line 240, in findall
    return _compile(pattern, flags).findall(string)
TypeError: expected string or bytes-like object

Expected behavior

Can be downloaded successfully by specifying DynamicURL or MasterURL.

Describe alternatives you've considered

Since this ability to specify DynamicURL and MasterURL does not appear to be used, I suggest that it be removed.
I will explain why I thought the function to specify them was not being used.

twspace-dl/twspace_dl/__main__.py

Lines 71 to 78 in c3552d3

else:
twspace = Twspace.from_space_url(args.input_url)
twspace_dl = TwspaceDL(twspace, args.output)
if args.from_dynamic_url:
twspace_dl.dyn_url = args.from_dynamic_url
if args.from_master_url:
twspace_dl.master_url = args.from_master_url

Reading the source code, one can see that the TwspaceDL class is called before DynamicURL and MasterURL are checked.

twspace-dl/twspace_dl/twspace_dl.py

Lines 32 to 70 in c3552d3

@cached_property
def dyn_url(self) -> str:
"""Returns the dynamic url i.e. the url used by the browser"""
space = self.space
if space["state"] == "Ended" and not space["available_for_replay"]:
logging.error(
(
"Can't Download. Space has ended, can't retrieve master url. "
"You can provide it with -f URL if you have it."
)
)
raise ValueError("Space Ended")
headers = {
"authorization": (
"Bearer "
"AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs"
"=1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA"
),
"cookie": "auth_token=",
}
media_key = space["media_key"]
response = requests.get(
"https://twitter.com/i/api/1.1/live_video_stream/status/" + media_key,
headers=headers,
)
try:
metadata = response.json()
except Exception as err:
raise RuntimeError("Space isn't available", space.source) from err
dyn_url = metadata["source"]["location"]
return dyn_url
@cached_property
def master_url(self) -> str:
"""Master URL for a space"""
master_url = re.sub(
r"(?<=/audio-space/).*", "master_playlist.m3u8", self.dyn_url
)
return master_url

Also, twspace_dl.dyn_url and twspace_dl.master_url are cached_property, so setters cannot be created.

Desktop (please complete the following information):

  • OS: macOS
  • Version: 12.3
  • Installation method: Build with poetry.
Ryu1845 commented

That is #52. Closed as duplicate

Ryu1845 commented

The master url and dynamic url can be used with a space url if a space has ended given you provide both the space url and the dynamic / master url

  • 👍1