레진 코믹스 다운로드 완료 후 언셔플 하는 과정에서 에러가 발생합니다.
Closed this issue · 7 comments
maeng-jin commented
/Users/maeng/Documents/webtoon/venv/bin/python /Users/maeng/Documents/webtoon/main.py
skipping 외전 ： 륜의 꽃나비연 5화: 99%|█████████▊| 541/549 [49:14<00:43, 5.46s/it]
Traceback (most recent call last):
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
response = conn.getresponse()
^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/connection.py", line 461, in getresponse
httplib_response = super().getresponse()
^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1378, in getresponse
response.begin()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 318, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/socket.py", line 706, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1278, in recv_into
return self.read(nbytes, buffer)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1134, in read
return self._sslobj.read(len, buffer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: The read operation timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 470, in increment
raise reraise(type(error), error, _stacktrace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
raise value
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 538, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='www.lezhin.com', port=443): Read timed out. (read timeout=30)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/maeng/Documents/webtoon/main.py", line 3, in <module>
wt.download_webtoon('cheonjihae', wt.L, authkey=authkey) # 첫 번째로 복사했던 수를 dr_hearthstone의 위치에 붙여넣으세요.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/WebtoonScraper/webtoon.py", line 183, in download_webtoon
webtoon_scraper.download_webtoon(episode_no_range, merge_amount=merge_amount)
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/A_scraper.py", line 334, in download_webtoon
self.download_episode(episode_no, webtoon_directory)
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/A_scraper.py", line 406, in download_episode
episode_images_url = self.get_episode_image_urls(episode_no)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_comics.py", line 255, in get_episode_image_urls
images_data = self.requests.get(images_retrieve_url).json()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/resoup/api_with_tools.py", line 133, in get
return request("get", url, params=params, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/resoup/api_with_tools.py", line 106, in request
response = ResponseProxy(session.request(method=method, url=url, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/maeng/Documents/webtoon/venv/lib/python3.11/site-packages/requests/adapters.py", line 532, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='www.lezhin.com', port=443): Read timed out. (read timeout=30)
혹시 어떤 이유인지 알 수 있을까요..?
ilotoki0804 commented
우선 CLI에 python -c "import WebtoonScraper;print(WebtoonScraper.__version__)"라고 쳐서 버전을 확인해 주실 수 있나요? 만약 최신 버전이 아니라면 우선 pip install -U WebtoonScraper로 업데이트 한 뒤에 다시 작동하는지 확인 부탁드립니다.
ilotoki0804 commented
maeng-jin commented
앗 빠른 대응 감사합니다!
그런데
pip install -U WebtoonScraper resoup
모두 진행 후 다시 진행 해보았는데
마찬가지로 아직 되지 않고 있네요 주륵..
실행 환경은 애플 실리콘 맥북(m1,m2)이며
파이썬 버전은 python-3.11.4-macos11
m1, m2 전부 동일한 에러를 뱉고 있습니다..
downloading 외전 ： 륜의 꽃나비연 終: 100%|██████████| 549/549 [54:16<00:00, 5.93s/it]
A webtoon 천지해 download ended.
Traceback (most recent call last):
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/main.py", line 4, in <module>
wt.download_webtoon('cheonjihae', wt.L, authkey=authkey) # 첫 번째로 복사했던 수를 dr_hearthstone의 위치에 붙여넣으세요.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/webtoon.py", line 190, in download_webtoon
webtoon_scraper.download_webtoon(episode_no_range, merge_amount=merge_amount)
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/A_scraper.py", line 319, in download_webtoon
webtoon_directory = self.unshuffle_lezhin_webtoon(webtoon_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_comics.py", line 280, in unshuffle_lezhin_webtoon
target_webtoon_directory = unshuffle_typical_webtoon_directory_and_return_target_directory(base_webtoon_directory, self.episode_int_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_unshuffler.py", line 26, in unshuffle_typical_webtoon_directory_and_return_target_directory
unshuffle_webtoon_directory_to_directory(source_webtoon_directory, target_webtoon_directory, episode_int_ids)
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_unshuffler.py", line 51, in unshuffle_webtoon_directory_to_directory
raise DirectoryStateUnmatchedError(f'Directory state is {directory_state}, which is not supported.')
WebtoonScraper.exceptions.DirectoryStateUnmatchedError: Directory state is not_matched, which is not supported.
- 혹시 이미 회차 이미지는 전부 다운 받은 상태에서 다시 실행하게 되면
해당 웹툰이 회차 수가 많아서 그런지 스킵 처리 되더라도 거진 30~50분의 시간을 잡아 먹게 되는데
다운로드 진행 회차 수를 지정하거나, 언셔플 하는 기능만 따로 실행하는 방법은 없을까요?
ilotoki0804 commented
방금 전에 2.3.3을 릴리즈했습니다. 제 컴퓨터 상에서는 문제 없이 작동하는 것을 확인했습니다. pip install -U WebtoonScraper로 업데이트하고 해결되었는지 확인해 주실 수 있나요?
- 혹시 이미 회차 이미지는 전부 다운 받은 상태에서 다시 실행하게 되면
해당 웹툰이 회차 수가 많아서 그런지 스킵 처리 되더라도 거진 30~50분의 시간을 잡아 먹게 되는데
다운로드 진행 회차 수를 지정하거나, 언셔플 하는 기능만 따로 실행하는 방법은 없을까요?
에피소드 다운로드 범위를 설정하면 불필요하게 이미 다운로드한 회차를 확인하는 과정을 스킵할 수 있습니다.
from WebtoonScraper import webtoon as wt
authkey = 'Bearer ...'
# 500화부터 끝까지 다운로드받고 싶은 경우 (500화(혹은 그 이상)까지 에피소드가 이미 완전히 다운로드되어 있을 경우 사용할 수 있음)
wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=(500, None))
# 모두 다운로드되어서 언셔플만 진행하고 싶은 경우 (1화만 다운로드받거나 스킵하고 다음 스텝 진행)
wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=1)maeng-jin commented
다시 한번 빠른 대응에 감사드립니다!
추가 답변 주신 회차 타겟 코드는 정상 동적하는데
마찬가지로 제가 가진 환경에서는 언셔플 과정이 제대로 동작하지 않네요 ㅠㅠ..
/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/bin/python /Users/bagmyeongjin/PycharmProjects/pythonProject1/main.py
Checking these platforms: lezhin
Webtoon's platform is assumed to be lezhin
downloading 외전 ： 륜의 꽃나비연 5화: 100%|██████████| 2/2 [00:10<00:00, 5.13s/it]
A webtoon 천지해 download ended.
Unshuffling is started. It takes a while and it's very CPU-intensive task. So keep patient and wait until the process end.
0%| | 0/2 [00:00<?, ?it/s]Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
skipping 외전 ： 륜의 꽃나비연 5화: 100%|██████████| 2/2 [00:07<00:00, 3.82s/it]
A webtoon 천지해 download ended.
Unshuffling is started. It takes a while and it's very CPU-intensive task. So keep patient and wait until the process end.
skipping 외전 ： 륜의 꽃나비연 4화: 50%|█████ | 1/2 [00:07<00:07, 7.69s/it]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 120, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 129, in _main
prepare(preparation_data)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 240, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 291, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen runpy>", line 291, in run_path
File "<frozen runpy>", line 98, in _run_module_code
File "<frozen runpy>", line 88, in _run_code
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/main.py", line 6, in <module>
wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=(540, 541))
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/webtoon.py", line 190, in download_webtoon
webtoon_scraper.download_webtoon(episode_no_range, merge_amount=merge_amount)
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/A_scraper.py", line 319, in download_webtoon
webtoon_directory = self.unshuffle_lezhin_webtoon(webtoon_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_comics.py", line 280, in unshuffle_lezhin_webtoon
target_webtoon_directory = unshuffle_typical_webtoon_directory_and_return_target_directory(base_webtoon_directory, self.episode_int_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_unshuffler.py", line 26, in unshuffle_typical_webtoon_directory_and_return_target_directory
unshuffle_webtoon_directory_to_directory(source_webtoon_directory, target_webtoon_directory, episode_int_ids)
File "/Users/bagmyeongjin/PycharmProjects/pythonProject1/venv/lib/python3.11/site-packages/WebtoonScraper/scrapers/J_lezhin_unshuffler.py", line 70, in unshuffle_webtoon_directory_to_directory
with multiprocessing.Pool(process_number) as p:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 215, in __init__
self._repopulate_pool()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 306, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 329, in _repopulate_pool_static
w.start()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/context.py", line 288, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 42, in _launch
prep_data = spawn.get_preparation_data(process_obj._name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 158, in get_preparation_data
_check_not_importing_main()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 138, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
우선 맥 환경에서는 이런 식의 에러 코드가 무한 루프 되고 있고요..
혹시 싶어 윈도우 환경에서 시도 해봤는데
C:\Users\maeng\PycharmProjects\pythonProject1\venv\Scripts\python.exe C:\Users\maeng\PycharmProjects\pythonProject1\main.py
Checking these platforms: lezhin
Webtoon's platform is assumed to be lezhin
downloading 외전 ： 륜의 꽃나비연 5화: 100%|██████████| 2/2 [00:12<00:00, 6.01s/it]
A webtoon 천지해 download ended.
Unshuffling is started. It takes a while and it's very CPU-intensive task. So keep patient and wait until the process end.
0%| | 0/2 [00:00<?, ?it/s]Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Checking these platforms: lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
0%| | 0/2 [00:00<?, ?it/s]Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
Webtoon's platform is assumed to be lezhin
skipping 외전 ： 륜의 꽃나비연 5화: 100%|██████████| 2/2 [00:08<00:00, 4.35s/it]
A webtoon 천지해 download ended.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 120, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 129, in _main
prepare(preparation_data)
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 240, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 291, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen runpy>", line 291, in run_path
File "<frozen runpy>", line 98, in _run_module_code
File "<frozen runpy>", line 88, in _run_code
File "C:\Users\maeng\PycharmProjects\pythonProject1\main.py", line 6, in <module>
wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=(540, 541))
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\webtoon.py", line 190, in download_webtoon
webtoon_scraper.download_webtoon(episode_no_range, merge_amount=merge_amount)
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\A_scraper.py", line 319, in download_webtoon
webtoon_directory = self.unshuffle_lezhin_webtoon(webtoon_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_comics.py", line 280, in unshuffle_lezhin_webtoon
target_webtoon_directory = unshuffle_typical_webtoon_directory_and_return_target_directory(base_webtoon_directory, self.episode_int_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_unshuffler.py", line 26, in unshuffle_typical_webtoon_directory_and_return_target_directory
unshuffle_webtoon_directory_to_directory(source_webtoon_directory, target_webtoon_directory, episode_int_ids)
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_unshuffler.py", line 37, in unshuffle_webtoon_directory_to_directory
ids_file_search_result = search_episode_int_ids_exclude_if_from_directory(source_webtoon_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_unshuffler.py", line 93, in search_episode_int_ids_exclude_if_from_directory
os.rename(id_text_file_source_path, id_text_file_target_path)
FileExistsError: [WinError 183] 파일이 이미 있으므로 만들 수 없습니다: 'webtoon\\천지해(cheonjihae, shuffled)\\cheonjihae_ids.txt' -> 'webtoon\\cheonjihae_ids.txt'
Checking these platforms: lezhin
Webtoon's platform is assumed to be lezhin
skipping 외전 ： 륜의 꽃나비연 5화: 100%|██████████| 2/2 [00:09<00:00, 4.96s/it]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 120, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 129, in _main
prepare(preparation_data)
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 240, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\maeng\AppData\Local\Programs\Python\Python311\Lib\multiprocessing\spawn.py", line 291, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen runpy>", line 291, in run_path
File "<frozen runpy>", line 98, in _run_module_code
File "<frozen runpy>", line 88, in _run_code
File "C:\Users\maeng\PycharmProjects\pythonProject1\main.py", line 6, in <module>
wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=(540, 541))
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\webtoon.py", line 190, in download_webtoon
webtoon_scraper.download_webtoon(episode_no_range, merge_amount=merge_amount)
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\A_scraper.py", line 319, in download_webtoon
webtoon_directory = self.unshuffle_lezhin_webtoon(webtoon_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_comics.py", line 280, in unshuffle_lezhin_webtoon
target_webtoon_directory = unshuffle_typical_webtoon_directory_and_return_target_directory(base_webtoon_directory, self.episode_int_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_unshuffler.py", line 26, in unshuffle_typical_webtoon_directory_and_return_target_directory
unshuffle_webtoon_directory_to_directory(source_webtoon_directory, target_webtoon_directory, episode_int_ids)
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_unshuffler.py", line 37, in unshuffle_webtoon_directory_to_directory
ids_file_search_result = search_episode_int_ids_exclude_if_from_directory(source_webtoon_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\maeng\PycharmProjects\pythonProject1\venv\Lib\site-packages\WebtoonScraper\scrapers\J_lezhin_unshuffler.py", line 93, in search_episode_int_ids_exclude_if_from_directory
os.rename(id_text_file_source_path, id_text_file_target_path)
윈도우 환경에서는 이런 식의 에러가 무한 루프 하고 있습니다 주륵..
ilotoki0804 commented
.py 파일을 통해 다운로드받을 경우 우선 해당 폴더에 있는 cheonjihae_ids.txt를 삭제하고 다음과 같이 코드를 작성하시면 됩니다.
from WebtoonScraper import webtoon as wt
if __name__ == "__main__":
authkey = 'Bearer ...'
wt.download_webtoon("cheonjihae", wt.LEZHIN, episode_no_range=1, authkey=authkey)혹은 Python 셸이나 Jupyter Notebook, CLI를 사용할 수도 있습니다.
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37)
Type "help", "copyright", "credits" or "license" for more information.
>>> from WebtoonScraper import webtoon as wt
>>> authkey = 'Bearer ...'
>>> wt.download_webtoon("cheonjihae", wt.LEZHIN, episode_no_range=1, authkey=authkey)
skipping 이야기의 시작: 100%|████████████████████████████████████████████████████████████████████| 1/1 [00:04<00:00, 4.01s/it]
A webtoon 천지해 download ended.
Unshuffling is started. It takes a while and it's very CPU-intensive task. So keep patient and wait until the process end.
Episode 0001. 이야기의 시작 unshuffle ended: 100%|███████████████████████████████████████████████| 1/1 [00:01<00:00, 1.31s/it] In [1]: from WebtoonScraper import webtoon as wt
...: authkey = 'Bearer ...'
...: wt.download_webtoon("cheonjihae", wt.LEZHIN, episode_no_range=1, authkey=authkey)
skipping 이야기의 시작: 100%|████████████████████████████████████████████████████████████████████| 1/1 [00:07<00:00, 7.32s/it]
A webtoon 천지해 download ended.
Unshuffling is started. It takes a while and it's very CPU-intensive task. So keep patient and wait until the process end.
Episode 0001. 이야기의 시작 unshuffle ended: 100%|███████████████████████████████████████████████| 1/1 [00:01<00:00, 1.27s/it](.venv) C:\Users>webtoon download cheonjihae -p lezhin --range 1 --authkey "Bearer ..."
Downloading has started(mock=False, webtoon_id='cheonjihae', platform='lezhin', merge_amount=None, cookie=None, authkey='Bearer ...', range=1, download_directory=WindowsPath('webtoon'), list_episodes=False, get_paid_episode=False, subparser_name='download').
skipping 이야기의 시작: 100%|████████████████████████████████████████████████████████████████████| 1/1 [00:05<00:00, 5.22s/it]
A webtoon 천지해 download ended.
Unshuffling is started. It takes a while and it's very CPU-intensive task. So keep patient and wait until the process end.
Episode 0001. 이야기의 시작 unshuffle ended: 100%|███████████████████████████████████████████████| 1/1 [00:01<00:00, 1.25s/it]언셔플러는 속도를 향상시키기 위해 multiprocessing을 이용합니다. 따라서 몇 가지 이상한 오류가 발생할 수 있습니다. WebtoonScraper의 오류는 아니며, 파이썬의 한계입니다.
maeng-jin commented
앗 말씀해주신 방법으로 해결하였습니다.
from WebtoonScraper import webtoon as wt
# 원래 있었던 코드를 전부 이 위에 올리세요.
authkey = 'Bearer..' # Bearer ...
# 500화부터 끝까지 다운로드받고 싶은 경우 (500화(혹은 그 이상)까지 에피소드가 이미 완전히 다운로드되어 있을 경우 사용할 수 있음)
wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=(540, 541))
# 모두 다운로드되어서 언셔플만 진행하고 싶은 경우 (1화만 다운로드받거나 스킵하고 다음 스텝 진행)
# wt.download_webtoon('cheonjihae', authkey=authkey, episode_no_range=1)
# wt.download_webtoon('cheonjihae', wt.L, authkey=authkey) # 첫 번째로 복사했던 수를 dr_hearthstone의 위치에 붙여넣으세요.
감사합니다!