kahing/goofys

ReadDir inode leak causes memory leak

1978629634 opened this issue · 0 comments

What happened?

The memory of the goofys process keeps growing and cannot reclaim the memory via
sync;echo 3 > /proc/sys/vm/drop_caches;killall -USR1 goofys;
Use go pprof heap to analyze memory
image
The inode created by ReadDir takes up a lot of memory

How can we reproduce it

I wrote a simple go program to reproduce the problem

package main

import (
	"fmt"
	"os"
)

func main() {
	_, err := os.ReadDir(os.Args[1])
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(os.Args[1])
}

Create 1000 files in the /mnt/goofys/test directory
Repeat run

go run main.go /mnt/goofys/test
sync;echo 3 > /proc/sys/vm/drop_caches;killall -USR1 goofys;

goofys log

/root/goofys: main.INFO Received user defined signal 1
/root/goofys: main.INFO forgot xxx inodes
/root/goofys: main.INFO 1004 inodes

/root/goofys: main.INFO Received user defined signal 1
/root/goofys: main.INFO forgot xxx inodes
/root/goofys: main.INFO 2003 inodes

/root/goofys: main.INFO Received user defined signal 1
/root/goofys: main.INFO forgot xxx inodes
/root/goofys: main.INFO 3006 inodes
...

The number of unrecyclable inodes will continue to increase

Problem causes

goofys ReadDir will cache all child inodes, but OS only caches the inodes of the parent directory. When memory is tight or echo 3 > drop_caches. OS call goofys to forget the parent inode. At this time, all child nodes will become dirty data exist in The fs.inodes map cannot be recycled.

What did you expect to happen?

sync;echo 3 > /proc/sys/vm/drop_caches;killall -USR1 goofys;can reclaim the memory