raspberry pi 3: out of memory when decrypting the master key
acavalin opened this issue · 10 comments
Hello,
thank you for this wonderful software :)
I encountered a problem using it on a raspberry pi 3: using a password 68 characters long with 23 unique characters, gocryptfs goes out of memory when decrypting the master key, it wants to allocate 1GiB but the rpi3 has not enough memory.
I tried adding a big swapfile but it still fails to allocate the memory. :'(
Is it possible to workaround this problem or the only solution left is shortening the length of the password?
# gocryptfs --version
gocryptfs v1.7; go-fuse v1.0.0-12-g0074c95; 2019-05-10 go1.7.4
# gocryptfs -passfile pass -reverse plain enc
passfile: reading from file "pass"
Decrypting master key
runtime: out of memory: cannot allocate 1073741824-byte block (1048576 in use)
fatal error: out of memory
runtime stack:
runtime.throw(0x245099, 0xd)
/usr/lib/go-1.7/src/runtime/panic.go:566 +0x78
runtime.largeAlloc(0x40000000, 0x1066e001, 0x83aa0)
/usr/lib/go-1.7/src/runtime/malloc.go:776 +0xc8
runtime.mallocgc.func1()
/usr/lib/go-1.7/src/runtime/malloc.go:669 +0x34
runtime.systemstack(0x336100)
/usr/lib/go-1.7/src/runtime/asm_arm.s:247 +0x80
runtime.mstart()
/usr/lib/go-1.7/src/runtime/proc.go:1079
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/lib/go-1.7/src/runtime/asm_arm.s:192 +0x4 fp=0x1064d83c sp=0x1064d838
runtime.mallocgc(0x40000000, 0x213c98, 0xdcb01, 0x1068f000)
/usr/lib/go-1.7/src/runtime/malloc.go:670 +0xe58 fp=0x1064d8dc sp=0x1064d83c
runtime.makeslice(0x213c98, 0x10000000, 0x0, 0x10000000, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go-1.7/src/runtime/slice.go:57 +0x130 fp=0x1064d904 sp=0x1064d8dc
golang.org/x/crypto/scrypt.Key(0x106ae000, 0x45, 0x802, 0x10673680, 0x20, 0x21, 0x100000, 0x8, 0x1, 0x20, ...)
golang.org/x/crypto/scrypt/scrypt.go:236 +0x194 fp=0x1064d990 sp=0x1064d904
github.com/rfjakob/gocryptfs/internal/configfile.(*ScryptKDF).DeriveKey(0x106a6074, 0x106ae000, 0x45, 0x802, 0x0, 0x0, 0x0)
github.com/rfjakob/gocryptfs/internal/configfile/scrypt.go:67 +0x94 fp=0x1064d9fc sp=0x1064d990
github.com/rfjakob/gocryptfs/internal/configfile.(*ConfFile).DecryptMasterKey(0x106a6060, 0x106ae000, 0x45, 0x802, 0x0, 0x0, 0x0, 0x0, 0x0)
github.com/rfjakob/gocryptfs/internal/configfile/config_file.go:218 +0x74 fp=0x1064da74 sp=0x1064d9fc
main.loadConfig(0x106a4000, 0x0, 0x0, 0x0, 0x106a6060, 0x0, 0x0)
github.com/rfjakob/gocryptfs/main.go:59 +0x2a0 fp=0x1064dad4 sp=0x1064da74
main.getMasterKey(0x106a4000, 0x0, 0x0, 0x0, 0x10678cd0)
github.com/rfjakob/gocryptfs/masterkey.go:64 +0x2a4 fp=0x1064db28 sp=0x1064dad4
main.initFuseFrontend(0x106a4000, 0x0, 0x0, 0x1064dcd0)
github.com/rfjakob/gocryptfs/mount.go:218 +0x48 fp=0x1064dc08 sp=0x1064db28
main.doMount(0x106a4000)
github.com/rfjakob/gocryptfs/mount.go:101 +0x824 fp=0x1064dd3c sp=0x1064dc08
main.main()
github.com/rfjakob/gocryptfs/main.go:297 +0xd04 fp=0x1064df9c sp=0x1064dd3c
runtime.main()
/usr/lib/go-1.7/src/runtime/proc.go:183 +0x264 fp=0x1064dfc4 sp=0x1064df9c
runtime.goexit()
/usr/lib/go-1.7/src/runtime/asm_arm.s:998 +0x4 fp=0x1064dfc4 sp=0x1064dfc4
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/lib/go-1.7/src/runtime/asm_arm.s:998 +0x4
goroutine 20 [chan send]:
github.com/rfjakob/gocryptfs/internal/cryptocore.(*randPrefetcherT).refillWorker(0x335fc8)
github.com/rfjakob/gocryptfs/internal/cryptocore/randprefetch.go:51 +0x54
created by github.com/rfjakob/gocryptfs/internal/cryptocore.init.1
github.com/rfjakob/gocryptfs/internal/cryptocore/randprefetch.go:16 +0x64
goroutine 21 [syscall]:
os/signal.signal_recv(0x0)
/usr/lib/go-1.7/src/runtime/sigqueue.go:116 +0x190
os/signal.loop()
/usr/lib/go-1.7/src/os/signal/signal_unix.go:22 +0x14
created by os/signal.init.1
/usr/lib/go-1.7/src/os/signal/signal_unix.go:28 +0x30
Hi, that's interesting. Master key decryption should only use 64MB of RAM, regardless of password length. Does a shorter password really fix this?
One more idea: you have compiled with a pretty old Go version, go1.7.4, can you try again with this, compiled with go1.12 ? gocryptfs_v1.7-33_arm.zip
Thank you for the more recent build, it runs without errors but it does not mount anything:
# ./gocryptfs -passfile pass -reverse plain enc
passfile: reading from file "pass"
Decrypting master key
# ls -l enc
total 0
I noticed that if re-initializing the folder on the rpi3 with the same password then everything goes well, but if I try to mount it using the original config file created on a x64 linux PC then it goes OOM... so it seems the problem is config related.
So I initialized some folders on the rpi3 and checked the differences versus the x64 config file, I discovered (ignoring EncryptedKey and Salt keys) that each file is identical except for the N key: in particular every config file created on the rpi3 has an N value of 65536 while the x64 one has 1048576.
I think the higher N value is the culprit 🤔
Yes, the N value explains it. So the question is: where does 1048576 come from?
What is your gocryptfs version on x64?
At that time I initialized the folder with gocryptfs v1.6.1.
What should I do now? Have I to reinitialize all folders?
It will be a pain to re-upload everything to my online drive but I will do it if necessary 😢
When you created it on x64, did you pass
-scryptn 20
? 2^20 = 1048576, so this is how you would get that value.
Grab the latest version from git (commit 991adfc ), then change your password like this:
gocryptfs -passwd -scryptn=16 CIPHERDIR
You can use the same password again. This will set scryptn=16, which is the default value, and then it will use 64MB.
Interesting riddle ;)