yandex-cloud/geesefs

slice bounds out of range during GetMultiReader

ctappy opened this issue · 6 comments

ctappy commented

Sep 28 22:11:58 linux geesefs[2458977]: 2023/09/28 22:11:58.279948 main.INFO I/O: 0.00 read/s, 0.00 % hits, 0.00 write/s; metadata: 0.00 read/s, 0.00 write/s, 0.00 noop/s, 1 alive, 0.00 evict/s; >Sep 28 22:11:59 linux geesefs[2458933]: panic: runtime error: slice bounds out of range [:18446744073709547520] with capacity 4096
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: goroutine 12366539 [running]:
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: github.com/yandex-cloud/geesefs/internal.(*Inode).GetMultiReader(0xc00049a800, 0x0, 0x0)
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: /home/runner/work/geesefs/geesefs/internal/file.go:1243 +0x94a
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: github.com/yandex-cloud/geesefs/internal.(*Inode).FlushSmallObject(0xc00049a800)
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: /home/runner/work/geesefs/geesefs/internal/file.go:1759 +0x485
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: created by github.com/yandex-cloud/geesefs/internal.(*Inode).SendUpload
Sep 28 22:11:59 dapu1kw3 geesefs[2458933]: /home/runner/work/geesefs/geesefs/internal/file.go:1522 +0x4a5

running 0.37.4, which happened while running Vim in Linux. I also had the same mount opened in Windows. I see there is an updated version in the csi-k8s release. I'll try that out.

not sure but there may be an additional check needed at

geesefs/internal/file.go

Line 1243 in af0ace5

reader.AddBuffer(b.data[0 : end-b.offset])

ctappy commented

seems to be working fine after the update to 0.38

ctappy commented

geesefs[778959]: panic: runtime error: slice bounds out of range [:18446744073709547520] with capacity 12288
geesefs[778959]: goroutine 19849 [running]:
geesefs[778959]: github.com/yandex-cloud/geesefs/internal.(*Inode).GetMultiReader(0xc000014800, 0x0, 0x0)
geesefs[778959]: /home/runner/work/geesefs/geesefs/internal/file.go:1243 +0x94a
geesefs[778959]: github.com/yandex-cloud/geesefs/internal.(*Inode).FlushSmallObject(0xc000014800)
geesefs[778959]: /home/runner/work/geesefs/geesefs/internal/file.go:1759 +0x485
geesefs[778959]: created by github.com/yandex-cloud/geesefs/internal.(*Inode).SendUpload
geesefs[778959]: /home/runner/work/geesefs/geesefs/internal/file.go:1522 +0x4a5
systemd[1]: geesefs-k8s.service: Main process exited, code=exited, status=2/INVALIDARGUMENT

same issue again, this time I was running mv before the issue happened.

geesefs 0.38.3 using k8s-csi-s3

ctappy commented 
➜  geesefs git:(v0.38.3) ✗ git status
HEAD detached at v0.38.3
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   internal/file.go
 
no changes added to commit (use "git add" and/or "git commit -a")
➜  geesefs git:(v0.38.3) ✗ git diff internal/file.go | cat
diff --git a/internal/file.go b/internal/file.go
index 30a943b..ca255b5 100644
--- a/internal/file.go
+++ b/internal/file.go
@@ -1240,6 +1240,9 @@ func (inode *Inode) GetMultiReader(offset uint64, size uint64) (reader *MultiRea
                        if b.zero {
                                reader.AddZero(end-b.offset)
                        } else {
+                               if end-b.offset > uint64(cap(b.data)) {
+                                       fmt.Println("Error with file: ", inode.Name)
+                               }
                                reader.AddBuffer(b.data[0 : end-b.offset])
                        }
                        break

I added the above and currently trying to debug it/easily recreate the issue

vitalif commented

Please retry with 0.40.0, all this buffer-juggling code has been refactored in it.

ctappy commented

I have debug up at least, this is not 0.40.0 and not my personal debug version. it's been fine for a while. I'll update to 0.40.0 and reopen if I see this. Thanks @vitalif!

ctappy commented 
Jan 14 20:41:00 node1kw6 geesefs[2555154]: 2024/01/14 20:41:00.993271 fuse.DEBUG Op 0x008395fe        connection.go:428] <- Unlink (parent 5432, name "%home%exampleuser%dev%scalaConversion%convertAmazon.scala.swp", PID 851796)
Jan 14 20:41:00 node1kw6 geesefs[2555154]: 2024/01/14 20:41:00.993313 fuse.DEBUG Unlink .local/share/nvim/swap/%home%exampleuser%dev%scalaConversion%convertAmazon.scala.swp
Jan 14 20:41:00 node1kw6 geesefs[2555154]: 2024/01/14 20:41:00.993342 fuse.DEBUG DeRef 79047 .local/share/nvim/swap/%home%exampleuser%dev%scalaConversion%convertAmazon.scala.swp [1 2]
Jan 14 20:41:00 node1kw6 geesefs[2555154]: 2024/01/14 20:41:00.993363 fuse.DEBUG Op 0x008395fe        connection.go:517] -> OK ()
Jan 14 20:41:00 node1kw6 geesefs[2555154]: panic: runtime error: slice bounds out of range [:18446744073709547520] with capacity 8192
Jan 14 20:41:00 node1kw6 geesefs[2555154]: goroutine 4448473 [running]:
Jan 14 20:41:00 node1kw6 geesefs[2555154]: github.com/yandex-cloud/geesefs/internal.(*Inode).GetMultiReader(0xc0028ff600, 0x0, 0x0)
Jan 14 20:41:00 node1kw6 geesefs[2555154]:         /home/runner/work/geesefs/geesefs/internal/file.go:1243 +0x94a
Jan 14 20:41:00 node1kw6 geesefs[2555154]: github.com/yandex-cloud/geesefs/internal.(*Inode).FlushSmallObject(0xc0028ff600)
Jan 14 20:41:00 node1kw6 geesefs[2555154]:         /home/runner/work/geesefs/geesefs/internal/file.go:1759 +0x485
Jan 14 20:41:00 node1kw6 geesefs[2555154]: created by github.com/yandex-cloud/geesefs/internal.(*Inode).SendUpload
Jan 14 20:41:00 node1kw6 geesefs[2555154]:         /home/runner/work/geesefs/geesefs/internal/file.go:1522 +0x4a5
Jan 14 20:41:01 node1kw6 systemd[1]: geesefs-nodep_2dusr_2dd1_2d00000005.service: Main process exited, code=exited, status=2/INVALIDARGUMENT