nlpodyssey/verbaflow

Memory climbs to >12GB while converting

mooijtech opened this issue · 4 comments

mooijtech commented

Memory climbs to >12GB using: ./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct convert before being killed by OS (out of memory).
Is this normal?

Thanks for your contributions :)

matteo-grella commented

What OS?

mooijtech commented

(Arch) Linux (x86_64)

matteo-grella commented

Thanks, let me investigate and I'll back to you beginning next week. Spoiler: we will move soon to https://github.com/nlpodyssey/safetensors which is much faster and requires less memory than the current pickle converter

  • 👍1
mooijtech commented

Possibly related memory stack-trace:

heap profile: 2: 6060038144 [4: 6060079216] @ heap/1048576
1: 6060032000 [1: 6060032000] @ 0x9f7ec7 0x9f4411 0x9f3fc5 0x9ec667 0x9eab57 0x9f3ab8 0x9f3565 0x9f34e5 0xb1212d 0xb15424 0xb15999 0xb0ff4f 0xb110aa 0xad2bab 0xad2e0d 0xacfa16 0xb0faf2 0xb0facb 0x43b227 0x46e461
#	0x9f7ec6	github.com/nlpodyssey/gopickle/pytorch.(*BFloat16Storage).SetFromFileWithSize+0x46	/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/storage.go:395
#	0x9f4410	github.com/nlpodyssey/gopickle/pytorch.loadTensor+0x190					/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:127
#	0x9f3fc4	github.com/nlpodyssey/gopickle/pytorch.loadZipFile.func1+0x2a4				/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:99
#	0x9ec666	github.com/nlpodyssey/gopickle/pickle.loadBinPersId+0x46				/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pickle/pickle.go:428
#	0x9eab56	github.com/nlpodyssey/gopickle/pickle.(*Unpickler).Load+0xf6				/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pickle/pickle.go:102
#	0x9f3ab7	github.com/nlpodyssey/gopickle/pytorch.loadZipFile+0x4f7				/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:107
#	0x9f3564	github.com/nlpodyssey/gopickle/pytorch.LoadWithUnpickler+0x44				/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:40
#	0x9f34e4	github.com/nlpodyssey/gopickle/pytorch.Load+0x24					/home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:31
#	0xb1212c	github.com/nlpodyssey/verbaflow/rwkvlm.(*converter[...]).loadTorchModelParams+0x2c	/home/bot/Documents/2023/verbaflow/rwkvlm/converter.go:416
#	0xb15423	github.com/nlpodyssey/verbaflow/rwkvlm.(*converter[...]).run+0x1a3			/home/bot/Documents/2023/verbaflow/rwkvlm/converter.go:117
#	0xb15998	github.com/nlpodyssey/verbaflow/rwkvlm.ConvertPickledModelToRWKVLM[...]+0x518		/home/bot/Documents/2023/verbaflow/rwkvlm/converter.go:78
#	0xb0ff4e	main.convert+0x12e									/home/bot/Documents/2023/verbaflow/cmd/verbaflow/main.go:139
#	0xb110a9	main.main.func3+0x29									/home/bot/Documents/2023/verbaflow/cmd/verbaflow/main.go:65
#	0xad2baa	github.com/urfave/cli/v2.(*Command).Run+0x9ea						/home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/command.go:273
#	0xad2e0c	github.com/urfave/cli/v2.(*Command).Run+0xc4c						/home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/command.go:266
#	0xacfa15	github.com/urfave/cli/v2.(*App).RunContext+0x615					/home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/app.go:332
#	0xb0faf1	github.com/urfave/cli/v2.(*App).Run+0x5d1						/home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/app.go:309
#	0xb0faca	main.main+0x5aa										/home/bot/Documents/2023/verbaflow/cmd/verbaflow/main.go:99
#	0x43b226	runtime.main+0x206									/usr/lib/go/src/runtime/proc.go:250