Broken on illumos?

Question

Broken on illumos?

Opened this issue 3 years ago · 4 comments

Trying to track down some caching problems with navidrome navidrome/navidrome#1048
and found that pulling down master and running "go test" fails:

$ go version
go version go1.16.3 illumos/amd64
$ uname -a
SunOS pergamum 5.11 omnios-r151030-5bd7739fe4 i86pc i386 i86pc illumos
$ git clone https://github.com/karrick/godirwalk.git
Cloning into 'godirwalk'...
remote: Enumerating objects: 1141, done.
remote: Counting objects: 100% (30/30), done.
remote: Compressing objects: 100% (21/21), done.
Receiving objects:  99% (1130/1141)remote: Total 1141 (delta 13), reused 21 (delta 9), pack-reused 1111
Receiving objects: 100% (1141/1141), 255.23 KiB | 4.12 MiB/s, done.
Resolving deltas: 100% (617/617), done.
$ pushd godirwalk/
~/navidrome/godirwalk ~/navidrome
$ go test
--- FAIL: TestReadDirents (0.00s)
    --- FAIL: TestReadDirents/without_symlinks (0.00s)
        readdir_test.go:14: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
    --- FAIL: TestReadDirents/with_symlinks (0.00s)
        readdir_test.go:51: GOT: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestScanner (0.00s)
    --- FAIL: TestScanner/collect_names (0.00s)
        scandir_test.go:22: GOT: "aaaaaa\x03" (extra)
        scandir_test.go:22: WANT: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" (missing)
        scandir_test.go:22: GOT: "symlin\x03" (extra)
        scandir_test.go:22: WANT: "symlinks" (missing)
    --- FAIL: TestScanner/collect_dirents (0.00s)
        scandir_test.go:35: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkCompatibleWithFilepathWalk (0.00s)
    --- FAIL: TestWalkCompatibleWithFilepathWalk/test_root (0.00s)
        walk_test.go:79: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkSkipThis (0.00s)
    --- FAIL: TestWalkSkipThis/SkipThis (0.00s)
        walk_test.go:154: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkFollowSymbolicLinks (0.00s)
    walk_test.go:196: GOT: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestErrorCallback (0.00s)
    --- FAIL: TestErrorCallback/halt (0.00s)
        walk_test.go:239: unexpected error callback for /tmp/godirwalk-378882479/d0/symlinks: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory
    --- FAIL: TestErrorCallback/skipnode (0.00s)
        walk_test.go:271: unexpected error callback for /tmp/godirwalk-378882479/d0/symlinks: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory
--- FAIL: TestPostChildrenCallback (0.00s)
    walk_test.go:299: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
FAIL
drwx------
drwxrwxr-x /d0
-rwxrwxr-x /d0/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
drwxrwxr-x /d0/d1
-rwxrwxr-x /d0/d1/f2
-rwxrwxr-x /d0/f1
drwxrwxr-x /d0/skips
drwxrwxr-x /d0/skips/d2
-rwxrwxr-x /d0/skips/d2/f3
-rwxrwxr-x /d0/skips/d2/skip
-rwxrwxr-x /d0/skips/d2/z1
drwxrwxr-x /d0/skips/d3
-rwxrwxr-x /d0/skips/d3/f4
drwxrwxr-x /d0/skips/d3/skip
-rwxrwxr-x /d0/skips/d3/skip/f5
-rwxrwxr-x /d0/skips/d3/z2
drwxrwxr-x /d0/symlinks
drwxrwxr-x /d0/symlinks/d4
Lrwxrwxrwx /d0/symlinks/d4/toSD1 -> ../toD1
Lrwxrwxrwx /d0/symlinks/d4/toSF1 -> ../toF1
Lrwxrwxrwx /d0/symlinks/nothing -> ../f0
Lrwxrwxrwx /d0/symlinks/toAbs -> /tmp/godirwalk-378882479/d0/f1
Lrwxrwxrwx /d0/symlinks/toD1 -> ../d1
Lrwxrwxrwx /d0/symlinks/toF1 -> ../f1
exit status 1
FAIL    github.com/karrick/godirwalk    0.023s

That weird truncation of "aaaaaa" is exactly what we're seeing with navidrome

Answer 1 · 2021-04-22T22:07:52.000Z

Relevant to #31

Answer 2 · 2021-04-22T22:10:05.000Z

In answer to the "if the tests fail please provide the results of go test -v"

$ go test -v
=== RUN   TestDirent
=== RUN   TestDirent/file
=== RUN   TestDirent/directory
=== RUN   TestDirent/symlink
=== RUN   TestDirent/symlink/to_file
=== RUN   TestDirent/symlink/to_directory
--- PASS: TestDirent (0.00s)
    --- PASS: TestDirent/file (0.00s)
    --- PASS: TestDirent/directory (0.00s)
    --- PASS: TestDirent/symlink (0.00s)
        --- PASS: TestDirent/symlink/to_file (0.00s)
        --- PASS: TestDirent/symlink/to_directory (0.00s)
=== RUN   TestReadDirents
=== RUN   TestReadDirents/without_symlinks
    readdir_test.go:14: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
=== RUN   TestReadDirents/with_symlinks
    readdir_test.go:51: GOT: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestReadDirents (0.00s)
    --- FAIL: TestReadDirents/without_symlinks (0.00s)
    --- FAIL: TestReadDirents/with_symlinks (0.00s)
=== RUN   TestReadDirnames
--- PASS: TestReadDirnames (0.00s)
=== RUN   TestScanner
=== RUN   TestScanner/collect_names
    scandir_test.go:22: GOT: "aaaaaa\x03" (extra)
    scandir_test.go:22: WANT: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" (missing)
    scandir_test.go:22: GOT: "symlin\x03" (extra)
    scandir_test.go:22: WANT: "symlinks" (missing)
=== RUN   TestScanner/collect_dirents
    scandir_test.go:35: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
=== RUN   TestScanner/symlink_to_directory
--- FAIL: TestScanner (0.00s)
    --- FAIL: TestScanner/collect_names (0.00s)
    --- FAIL: TestScanner/collect_dirents (0.00s)
    --- PASS: TestScanner/symlink_to_directory (0.00s)
=== RUN   TestWalkCompatibleWithFilepathWalk
=== RUN   TestWalkCompatibleWithFilepathWalk/test_root
    walk_test.go:79: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
=== RUN   TestWalkCompatibleWithFilepathWalk/ignore_skips
--- FAIL: TestWalkCompatibleWithFilepathWalk (0.00s)
    --- FAIL: TestWalkCompatibleWithFilepathWalk/test_root (0.00s)
    --- PASS: TestWalkCompatibleWithFilepathWalk/ignore_skips (0.00s)
=== RUN   TestWalkSkipDir
=== RUN   TestWalkSkipDir/skip_file_at_root
=== RUN   TestWalkSkipDir/skip_dir_at_root
=== RUN   TestWalkSkipDir/skip_nodes_under_root
=== RUN   TestWalkSkipDir/SkipDirOnSymlink
--- PASS: TestWalkSkipDir (0.00s)
    --- PASS: TestWalkSkipDir/skip_file_at_root (0.00s)
    --- PASS: TestWalkSkipDir/skip_dir_at_root (0.00s)
    --- PASS: TestWalkSkipDir/skip_nodes_under_root (0.00s)
    --- PASS: TestWalkSkipDir/SkipDirOnSymlink (0.00s)
=== RUN   TestWalkSkipThis
=== RUN   TestWalkSkipThis/SkipThis
    walk_test.go:154: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkSkipThis (0.00s)
    --- FAIL: TestWalkSkipThis/SkipThis (0.00s)
=== RUN   TestWalkFollowSymbolicLinks
    walk_test.go:196: GOT: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestWalkFollowSymbolicLinks (0.00s)
=== RUN   TestErrorCallback
=== RUN   TestErrorCallback/halt
    walk_test.go:239: unexpected error callback for /tmp/godirwalk-286333511/d0/symlinks: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory
=== RUN   TestErrorCallback/skipnode
    walk_test.go:271: unexpected error callback for /tmp/godirwalk-286333511/d0/symlinks: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory
--- FAIL: TestErrorCallback (0.00s)
    --- FAIL: TestErrorCallback/halt (0.00s)
    --- FAIL: TestErrorCallback/skipnode (0.00s)
=== RUN   TestPostChildrenCallback
    walk_test.go:299: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestPostChildrenCallback (0.00s)
FAIL
drwx------
drwxrwxr-x /d0
-rwxrwxr-x /d0/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
drwxrwxr-x /d0/d1
-rwxrwxr-x /d0/d1/f2
-rwxrwxr-x /d0/f1
drwxrwxr-x /d0/skips
drwxrwxr-x /d0/skips/d2
-rwxrwxr-x /d0/skips/d2/f3
-rwxrwxr-x /d0/skips/d2/skip
-rwxrwxr-x /d0/skips/d2/z1
drwxrwxr-x /d0/skips/d3
-rwxrwxr-x /d0/skips/d3/f4
drwxrwxr-x /d0/skips/d3/skip
-rwxrwxr-x /d0/skips/d3/skip/f5
-rwxrwxr-x /d0/skips/d3/z2
drwxrwxr-x /d0/symlinks
drwxrwxr-x /d0/symlinks/d4
Lrwxrwxrwx /d0/symlinks/d4/toSD1 -> ../toD1
Lrwxrwxrwx /d0/symlinks/d4/toSF1 -> ../toF1
Lrwxrwxrwx /d0/symlinks/nothing -> ../f0
Lrwxrwxrwx /d0/symlinks/toAbs -> /tmp/godirwalk-286333511/d0/f1
Lrwxrwxrwx /d0/symlinks/toD1 -> ../d1
Lrwxrwxrwx /d0/symlinks/toF1 -> ../f1
exit status 1
FAIL    github.com/karrick/godirwalk    0.026s

Answer 3 · 2022-05-04T00:53:34.000Z

I wonder whether Illumos' file system file name limits are related? https://illumos.org/man/2/intro

Answer 4 · 2022-05-04T19:05:13.000Z

I am not sure which limits you are referring to
When godirwalk.Walk() was replaced with filepath.WalkDir() in navidrome , all of the unwanted filename truncation disappeared and cross-platform functionality was enabled.