validator corrupted on runtime

[ewagmig]

Hi there,
A validator has been established since 9 days ago, but corrupted recently. Some bug log is like below:

goroutine 30208 [select, 13840 minutes]:
github.com/QOSGroup/qbase/store.(*iavlIterator).iterateRoutine.func1(0xc0009ee220, 0x12, 0x12, 0xc0009ee280, 0x14, 0x14, 0x1135e10)
	/go/pkg/mod/github.com/!q!o!s!group/qbase@v0.1.0/store/iavlstore.go:323 +0x128
github.com/tendermint/iavl.(*ImmutableTree).IterateRange.func1(0xc0009fe2c0, 0x2, 0x12)
	/go/pkg/mod/github.com/tendermint/iavl@v0.12.0/immutable_tree.go:125 +0x65
github.com/tendermint/iavl.(*Node).traverseInRange(0xc0009fe2c0, 0xc00092c0a0, 0xc00630e7c0, 0xa, 0xa, 0xc00630e7d0, 0xa, 0xa, 0xc007020001, 0xc007f14f48, ...)
	/go/pkg/mod/github.com/tendermint/iavl@v0.12.0/node.go:390 +0xbb
github.com/tendermint/iavl.(*Node).traverseInRange(0xc0009fe210, 0xc00092c0a0, 0xc00630e7c0, 0xa, 0xa, 0xc00630e7d0, 0xa, 0xa, 0xc001010001, 0xc007f14f48, ...)
	/go/pkg/mod/github.com/tendermint/iavl@v0.12.0/node.go:402 +0x195
github.com/tendermint/iavl.(*Node).traverseInRange(0xc00567b4a0, 0xc00092c0a0, 0xc00630e7c0, 0xa, 0xa, 0xc00630e7d0, 0xa, 0xa, 0x1, 0xc007f14f48, ...)
	/go/pkg/mod/github.com/tendermint/iavl@v0.12.0/node.go:408 +0x259
github.com/tendermint/iavl.(*ImmutableTree).IterateRange(0xc00092c0a0, 0xc00630e7c0, 0xa, 0xa, 0xc00630e7d0, 0xa, 0xa, 0x1135b01, 0xc00159efb8, 0xc00159efc8)
	/go/pkg/mod/github.com/tendermint/iavl@v0.12.0/immutable_tree.go:123 +0xcd
github.com/QOSGroup/qbase/store.(*iavlIterator).iterateRoutine(0xc006ce7680)
	/go/pkg/mod/github.com/!q!o!s!group/qbase@v0.1.0/store/iavlstore.go:320 +0x95
created by github.com/QOSGroup/qbase/store.newIAVLIterator
	/go/pkg/mod/github.com/!q!o!s!group/qbase@v0.1.0/store/iavlstore.go:313 +0x18b

For more, please refer to the full log attched:

val_docker_error.zip