warp-contracts/warp

[BUG] Cache corruption

Closed this issue · 2 comments

balthazar commented

From time to time, we are experiencing what I would qualify a cache corruption, which seem to encompass those errors:

  • database disk image is malformed
  • file is not a database
  • disk I/O error
  • database is locked

In those scenarios, it seems the only way to get around it is to delete the cache and restart from scratch. Note that we do not run multiple evaluations against the same cache at once. It appears to happen pretty randomly, so I'm quite unsure as to what could be the cause of it, maybe the fact we run it on a NFS disk?

Specs:

  • OS: Amazon Linux 2
  • Node.js version: 18
  • Volume: NFS
  • warp-contracts SDK version: 1.4.12
  • plugins: warp-contracts-plugin-vm2@1.0.2 warp-contracts-sqlite@1.0.1
ppedziwiatr commented

Hey @balthazar , thanks for letting us know!
Could it be the case application that is using the cache is being stopped/aborted abnormally? i.e. due to some uncaught exception / unhandled promise rejection? do you gracefully shutdown the application? e.g. as we do in the DRE: https://github.com/warp-contracts/warp-dre-node/blob/main/src/listener.js#L361 - we're using async-exit-hook lib here.

balthazar commented

Hello PPE, yeah it might be due to an unclean exit, thanks for showing me this exit hook, will probably add that myself! Feel free to close the issue as it might be out-of-scope