Failed to process some files (absolute path longer than 255 bytes)
funnym0nk3y opened this issue · 8 comments
I tried your tool, it is awsome and helped me to save a lot of time! Thanks!
But I had an error with the following path:
Syncing .\test-ilias-downloader\Komplexe Analysis und Integraltransformationen (KAI)\Lehrvideos Komplexe Analysis und Integraltransformationen SS2020\Kapitel 2.6-2.7 Rücktransformation mit Faltungsregel, Rechenregeln der Laplace-Transformation.mp4: failed to process URL
Caused by:
0: failed to save video
1: failed to create file
2: Das System kann den angegebenen Pfad nicht finden. (os error 3)
Maybe you could look into that.
The problem is that the file path is longer than 255 characters. This is a limitation of the file system used by Windows (NTFS).
It is possible to create longer paths, but the resulting directory structure will not work correctly in some programs (including Explorer).
You can instead try to put the lectures on your "personal desktop":
The downloader will then create a new folder next to your courses.
NTFS should allow up to 32.767 characters UTF-16 with some API functions limiting it to 260 characters. However there is a function to enable long paths. The following must be included in the application manifest file (KB) :
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
<ws2:longPathAware>true</ws2:longPathAware>
</windowsSettings>
</application>
Could this be included?
The Microsoft docs state:
In editions of Windows before Windows 10 version 1607, the maximum length for a path is MAX_PATH,
which is defined as 260 characters. In later versions of Windows, changing a registry key
or using the Group Policy tool is required to remove the limit. See Maximum Path Length Limitation for full details.
I don't know whether it is possible to create longer paths if that setting is not changed. But I'll try to use long paths anyway.
As far as I understood, both the manifest and the registry key must be set. To begin with we could set up CI to include the manifest and add a paragraph in the wiki.
You can download a build that uses long paths (via \\?\) here: https://github.com/FliegendeWurst/KIT-ILIAS-downloader/suites/1889688550/artifacts/36955251
I didn't change the application manifest yet. winres is probably the best way to do that, if it is really neccessary.
Thanks! Also I like to contribute to this project, but please be patient with me, I'm still learning. I'd like to automate the release generation, so that for each new tag a new release gets generated.
Also I like to note that unfortunately this did not solve the problem.
Syncing \\?\C:\Users\funnym0nk3y\OneDrive - student.kit.edu\Studium\ILIAS_downloader_ingestion\Komplexe Analysis und Integraltransformationen (KAI)\Lehrvideos Komplexe Analysis und Integraltransformationen SS2020\Kapitel 2.6-2.7 Rücktransformation mit Faltungsregel, Rechenregeln der Laplace-Transformation.mp4: failed to process URL
Caused by:
0: error sending request for url (https://lb-bib.bibliothek.kit.edu/staticfiles/mh_default_org/api/fe51d354-d074-4b5b-9697-eec9f2d7c1be/675a4464-1757-4219-bae5-58494db56813/KAI_Vorlesung_Kap26_27_compressed.mp4?policy=eyJTdGF0ZW1lbnQiOnsiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6MTYxMTUxOTY5NDAwMH0sIlJlc291cmNlIjoiaHR0cHM6XC9cL29jLWRlbGl2ZXJ5LmJpYmxpb3RoZWsua2l0LmVkdVwvc3RhdGljZmlsZXNcL21oX2RlZmF1bHRfb3JnXC9hcGlcL2ZlNTFkMzU0LWQwNzQtNGI1Yi05Njk3LWVlYzlmMmQ3YzFiZVwvNjc1YTQ0NjQtMTc1Ny00MjE5LWJhZTUtNTg0OTRkYjU2ODEzXC9LQUlfVm9ybGVzdW5nX0thcDI2XzI3X2NvbXByZXNzZWQubXA0In19&keyId=default&signature=5f246e59bafc82e8a751d295d02883c3d667d70ac53473556823d037b8d02566): http2 error: protocol error: not a result of an error
1: http2 error: protocol error: not a result of an error
2: protocol error: not a result of an error
The HTTP/2 error is unrelated (check out seanmonstar/reqwest#853 + seanmonstar/reqwest#1052 if you want to know more about it).
The error can usually be fixed by simple running the downloader again. Perhaps some retry mechanism could be built into the downloader.