panic: runtime error: slice bounds out of range for archive node
Johnaverse opened this issue · 6 comments
version: go-opera/v1.1.2-txtracing-rc.6-f8983789-1691829635
The ftm height is 67571797 and the opera crashed with following error
Aug 29 13:06:18 fantom-arch46 opera[1510228]: panic: runtime error: slice bounds out of range [:-8628854311100398772]
Aug 29 13:06:18 fantom-arch46 opera[1510228]: goroutine 6952 [running]:
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/txtrace.(*TraceStructLogger).CaptureState(0xc01669d340, 0x0?, 0xc4b1ca4d72b62a64?, 0xf1, 0x698d4, 0x0?, 0>
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/txtrace/trace_logger.go:178 +0x2710
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/ethereum/go-ethereum/core/vm.(*EVMInterpreter).Run.func4()
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/build/pkg/mod/github.com/!fantom-foundation/go-ethereum@v1.10.8-ftm-rc11/core/vm/interpreter.go:174 +0x123
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/ethereum/go-ethereum/core/vm.(*EVMInterpreter).Run(0xc02aaaa900, 0xc02da45740, {0xc002244c30, 0x44, 0x44}, 0x0)
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/build/pkg/mod/github.com/!fantom-foundation/go-ethereum@v1.10.8-ftm-rc11/core/vm/interpreter.go:234 +0xe2f
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/ethereum/go-ethereum/core/vm.(*EVM).Call(0xc02aaa6a80, {0x19910e0, 0xc000493080}, {0x6e, 0x2f, 0x9b, 0x54, 0x48, 0x98, 0xf6, ...}, .>
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/build/pkg/mod/github.com/!fantom-foundation/go-ethereum@v1.10.8-ftm-rc11/core/vm/evm.go:237 +0xd05
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/evmcore.(*StateTransition).TransitionDb(0xc01a94d880)
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/evmcore/state_transition.go:292 +0x599
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/evmcore.ApplyMessage(0x168b2a0?, {0x19a2b08?, 0xc02aa80a20?}, 0xfc54f24db7c4adc2?)
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/evmcore/state_transition.go:173 +0x32
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/evmcore.applyTransaction({0xc000492f00, {0x0, 0x1a, 0x5, 0x55, 0x7b, 0xd0, 0xd9, 0x21, 0x2, ...}, ...}, .>
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/evmcore/state_processor.go:137 +0x4a8
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/evmcore.(*StateProcessor).Process(_, _, _, {0x1, {0x199c4e0, 0xc01669d340}, 0x0, 0x0, 0x0, {0x0, ...}, ..>
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/evmcore/state_processor.go:85 +0x60d
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/gossip/blockproc/evmmodule.(*OperaEVMProcessor).Execute(0xc02c932000, {0xc027efc0a0?, 0x6, 0xc0175923c0?})
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/gossip/blockproc/evmmodule/evm.go:91 +0x145
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/gossip.consensusCallbackBeginBlockFn.func1.3.4()
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/gossip/c_block_callbacks.go:322 +0x7ae
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/go-opera/gossip.consensusCallbackBeginBlockFn.func1.3.5()
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/go-opera/gossip/c_block_callbacks.go:469 +0x83
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/lachesis-base/utils/workers.worker(...)
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/build/pkg/mod/github.com/!fantom-foundation/lachesis-base@v0.0.0-20221208123620-82a6d15f995c/utils/worker>
Aug 29 13:06:18 fantom-arch46 opera[1510228]: github.com/Fantom-foundation/lachesis-base/utils/workers.(*Workers).Start.func1()
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/build/pkg/mod/github.com/!fantom-foundation/lachesis-base@v0.0.0-20221208123620-82a6d15f995c/utils/worker>
Aug 29 13:06:18 fantom-arch46 opera[1510228]: created by github.com/Fantom-foundation/lachesis-base/utils/workers.(*Workers).Start
Aug 29 13:06:18 fantom-arch46 opera[1510228]: /tmp/build.1692887881.1599549/build/pkg/mod/github.com/!fantom-foundation/lachesis-base@v0.0.0-20221208123620-82a6d15f995c/utils/worker>
Aug 29 13:06:18 fantom-arch46 systemd[1]: opera.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
My parms:
--genesis=<path>
--config=<path>
--port=5050
--maxpeers=200
--datadir=<path>
--http
--http.addr=0.0.0.0
--http.port=18545
--http.corsdomain=*
--http.vhosts=*
--ws
--ws.addr=0.0.0.0
--ws.port=18546
--ws.origins=*
--db.preset=pbl-1
--tracenode
--http.api=eth,web3,net,ftm,trace
--metrics
--metrics.addr=0.0.0.0
--metrics.port=6060
Same here using the same version.
It seems this has been fixed on the latest commit on that branch. Just not sure if it's safe to use
Yes, this issue is fixed in release/txtracing/1.1.2-rc.6, please use this updated branch and is safe to use. Node should recover itself.
FYI, we're still running smoothly on v1.1.2-rc.5 after we missed the upgrade to rc.6.
The latest hotfixes are now available in the following branches:
For version 1.1.2, refer to the branch release/txtracing/1.1.2-rc.6.1.
For version 1.1.3, check out release/txtracing/1.1.3-rc.3.1.
If you previously used the release/txtracing/1.1.2-rc.5 branch, we recommend upgrading directly to release/txtracing/1.1.2-rc.6.1.
While the earlier hotfix addressed this specific crash, there remained potential for other related crashes. These have now been addressed in the recent update.
If you experienced this crash, It's likely that your node DB is in now a dirty state and won't restart. You can execute the command opera --datadir /path/to/datadir db heal --experimental for recovery. When you restart your node, it will start the EVM snapshot reconstruction. This recovery process may be time-consuming, with the duration primarily influenced by your disk speed.