Path separator issue on Windows
JackDunnNZ opened this issue · 4 comments
JackDunnNZ commented
If there are mixed path separators in the path on Windows, then it can sometimes cause the specified file to not be found. For example, using the example in the test folder
using RelocatableFolders
PATH = @path "path"
FILE = joinpath("path", "subfolder/other.jl") # works
FILE = joinpath(PATH, "subfolder/other.jl") # doesn't find other.jl
FILE = joinpath(PATH, "subfolder\\other.jl") # works
FILE = joinpath(PATH, "subfolder", "other.jl") # worksThe cause is that the paths may not compare equal at this line due to differences in the path separators used in each path. An easy fix is to instead compare normpath(path) == normpath(fullpath), but not sure if that's the best solution
MichaelHatherly commented
Thanks for reporting this. I'm not able to reproduce the issue with the given example at the moment. By "works" and "doesn't find other.jl" do you mean that isfile(FILE) is returning a false for the second example, or that you're getting an error from it?
If you're able to make that example more reproducible that would be great. Could we also get versioninfo() and ] st RelocatableFolders output thanks? The output of dump(PATH) may be helpful in diagnosing this as well.
JackDunnNZ commented
Thanks for the response, and whoops, I totally messed up the example and left off the @path in each call to FILE, sorry about that. This should be fully reproducible from a fresh julia depot:
using RelocatableFolders
cd(joinpath(dirname(pathof(RelocatableFolders)), "../test"))
PATH = @path "path"
dump(PATH)
FILE = @path joinpath("path", "subfolder/other.jl")
dump(FILE)
FILE = @path joinpath(PATH, "subfolder/other.jl") # this one doesn't work, `FILE.files` is empty
dump(FILE)
FILE = @path joinpath(PATH, "subfolder\\other.jl")
dump(FILE)
FILE = @path joinpath(PATH, "subfolder", "other.jl")
dump(FILE)Output from my system:
julia> versioninfo()
Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: AMD Ryzen Threadripper 2970WX 24-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, znver1)
(@v1.6) pkg> add RelocatableFolders
Installing known registries into `C:\Users\iai\.julia`
Cloning registry from "https://github.com/JuliaRegistries/General.git"
Added registry `General` to `C:\Users\iai\.julia\registries\General`
Resolving package versions...
Installed Scratch ──────────── v1.1.0
Installed RelocatableFolders ─ v0.1.2
Updating `C:\Users\iai\.julia\environments\v1.6\Project.toml`
[05181044] + RelocatableFolders v0.1.2
Updating `C:\Users\iai\.julia\environments\v1.6\Manifest.toml`
[05181044] + RelocatableFolders v0.1.2
[6c6a2e73] + Scratch v1.1.0
[ade2ca70] + Dates
[de0858da] + Printf
[ea8e919c] + SHA
[4ec0a83e] + Unicode
Precompiling project...
2 dependencies successfully precompiled in 5 seconds
julia> using RelocatableFolders
julia> cd(joinpath(dirname(pathof(RelocatableFolders)), "../test"))
julia> PATH = @path "path"
"C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path"
julia> dump(PATH)
RelocatableFolders.Path
is_dir: Bool true
mod: Module Main
path: String "C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path"
hash: String "d306ff6859f4ea072957d2b4ef79899035feebb1"
files: Dict{String, Vector{UInt8}}
slots: Array{UInt8}((16,)) UInt8[0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x00, 0x00, 0x00]
keys: Array{String}((16,))
1: #undef
2: #undef
3: #undef
4: String "text.md"
5: #undef
...
12: String "file.jl"
13: String "subfolder\\other.jl"
14: #undef
15: #undef
16: #undef
vals: Array{Vector{UInt8}}((16,))
1: #undef
2: #undef
3: #undef
4: Array{UInt8}((7,)) UInt8[0x74, 0x65, 0x78, 0x74, 0x2e, 0x6d, 0x64]
5: #undef
...
12: Array{UInt8}((9,)) UInt8[0x23, 0x20, 0x66, 0x69, 0x6c, 0x65, 0x2e, 0x6a, 0x6c]
13: Array{UInt8}((10,)) UInt8[0x23, 0x20, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x2e, 0x6a, 0x6c]
14: #undef
15: #undef
16: #undef
ndel: Int64 0
count: Int64 3
age: UInt64 0x0000000000000003
idxfloor: Int64 1
maxprobe: Int64 0
julia> FILE = @path joinpath("path", "subfolder/other.jl")
"C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder\\other.jl"
julia> dump(FILE)
RelocatableFolders.Path
is_dir: Bool false
mod: Module Main
path: String "C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder"
hash: String "f7864905002a58658723784aac886712ef7ce302"
files: Dict{String, Vector{UInt8}}
slots: Array{UInt8}((16,)) UInt8[0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
keys: Array{String}((16,))
1: #undef
2: String "other.jl"
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
vals: Array{Vector{UInt8}}((16,))
1: #undef
2: Array{UInt8}((10,)) UInt8[0x23, 0x20, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x2e, 0x6a, 0x6c]
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
ndel: Int64 0
count: Int64 1
age: UInt64 0x0000000000000001
idxfloor: Int64 1
maxprobe: Int64 0
julia> FILE = @path joinpath(PATH, "subfolder/other.jl")
Error showing value of type RelocatableFolders.Path:
ERROR: ArgumentError: collection must be non-empty
Stacktrace:
[1] first
@ .\abstractarray.jl:387 [inlined]
[2] getroot(p::RelocatableFolders.Path, root::String) (repeats 2 times)
@ RelocatableFolders C:\Users\iai\.julia\packages\RelocatableFolders\2eEye\src\RelocatableFolders.jl:90
[3] show
@ C:\Users\iai\.julia\packages\RelocatableFolders\2eEye\src\RelocatableFolders.jl:69 [inlined]
[4] show(io::IOContext{Base.TTY}, #unused#::MIME{Symbol("text/plain")}, x::RelocatableFolders.Path)
@ Base.Multimedia .\multimedia.jl:47
[5] (::REPL.var"#38#39"{REPL.REPLDisplay{REPL.LineEditREPL}, MIME{Symbol("text/plain")}, Base.RefValue{Any}})(io::Any)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:220
[6] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:462
[7] display(d::REPL.REPLDisplay, mime::MIME{Symbol("text/plain")}, x::Any)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:213
[8] display(d::REPL.REPLDisplay, x::Any)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:225
[9] display(x::Any)
@ Base.Multimedia .\multimedia.jl:328
[10] #invokelatest#2
@ .\essentials.jl:708 [inlined]
[11] invokelatest
@ .\essentials.jl:706 [inlined]
[12] print_response(errio::IO, response::Any, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:247
[13] (::REPL.var"#40#41"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:231
[14] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:462
[15] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:229
[16] (::REPL.var"#do_respond#61"{Bool, Bool, REPL.var"#72#82"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:798
[17] #invokelatest#2
@ .\essentials.jl:708 [inlined]
[18] invokelatest
@ .\essentials.jl:706 [inlined]
[19] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
@ REPL.LineEdit C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\LineEdit.jl:2441
[20] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
@ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:1126
[21] (::REPL.var"#44#49"{REPL.LineEditREPL, REPL.REPLBackendRef})()
@ REPL .\task.jl:411
julia> dump(FILE)
RelocatableFolders.Path
is_dir: Bool false
mod: Module Main
path: String "C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder"
hash: String "da39a3ee5e6b4b0d3255bfef95601890afd80709"
files: Dict{String, Vector{UInt8}}
slots: Array{UInt8}((16,)) UInt8[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
keys: Array{String}((16,))
1: #undef
2: #undef
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
vals: Array{Vector{UInt8}}((16,))
1: #undef
2: #undef
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
ndel: Int64 0
count: Int64 0
age: UInt64 0x0000000000000000
idxfloor: Int64 1
maxprobe: Int64 0
julia> FILE = @path joinpath(PATH, "subfolder\\other.jl")
"C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder\\other.jl"
julia> dump(FILE)
RelocatableFolders.Path
is_dir: Bool false
mod: Module Main
path: String "C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder"
hash: String "f7864905002a58658723784aac886712ef7ce302"
files: Dict{String, Vector{UInt8}}
slots: Array{UInt8}((16,)) UInt8[0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
keys: Array{String}((16,))
1: #undef
2: String "other.jl"
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
vals: Array{Vector{UInt8}}((16,))
1: #undef
2: Array{UInt8}((10,)) UInt8[0x23, 0x20, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x2e, 0x6a, 0x6c]
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
ndel: Int64 0
count: Int64 1
age: UInt64 0x0000000000000001
idxfloor: Int64 1
maxprobe: Int64 0
julia> FILE = @path joinpath(PATH, "subfolder", "other.jl")
"C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder\\other.jl"
julia> dump(FILE)
RelocatableFolders.Path
is_dir: Bool false
mod: Module Main
path: String "C:\\Users\\iai\\.julia\\packages\\RelocatableFolders\\2eEye\\test\\path\\subfolder"
hash: String "f7864905002a58658723784aac886712ef7ce302"
files: Dict{String, Vector{UInt8}}
slots: Array{UInt8}((16,)) UInt8[0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
keys: Array{String}((16,))
1: #undef
2: String "other.jl"
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
vals: Array{Vector{UInt8}}((16,))
1: #undef
2: Array{UInt8}((10,)) UInt8[0x23, 0x20, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x2e, 0x6a, 0x6c]
3: #undef
4: #undef
5: #undef
...
12: #undef
13: #undef
14: #undef
15: #undef
16: #undef
ndel: Int64 0
count: Int64 1
age: UInt64 0x0000000000000001
idxfloor: Int64 1
maxprobe: Int64 0
MichaelHatherly commented
Cool, that did the trick. A normpath is probably needed, I think just doing it at the start of that function is probably best, i.e. https://github.com/JuliaPackaging/RelocatableFolders.jl/blob/main/src/RelocatableFolders.jl#L48 path = normpath(path) just to ensure it's normalised throughout the function. Would you like to PR that fix?
JackDunnNZ commented
Thanks, done!