DB Cache Size - improve description
Closed this issue · 2 comments
I have repeatedly encountered a problem during initial block download that after I have forcibly killed the bitcoind process then after restarting it it needs to reprocess blocks from disk and it takes a lot of time. I digged a bit deeper and it turns out that chainstate is committed to disk just after bitcoind runs out of db cache space in memory. So increasing cache to above 10 GiB means chainstate is never committed even after hundreds of thousands of blocks. And paradoxically the higher the cache size the longer the graceful shutdown takes to commit it all to disk and the higher chance it fails to commit.
The current description is confusing, it doesn't warn about the trade off. I'd argue that it is best to spend 0.1% - 1% of time to repeatedly commit the chainstate so the block processing doesn't need to be repeated after crash and especially on fast SSD (500 MiB/s) even cache as small as 512 MiB doesn't degrade IBD performance by more than 1%.
Based on my tests I'd suggest that even on slow HDD the maximum cache should be 2-4 GiB and on fast SSD 0.5 GiB - 1 GiB. The lower value when assuming the IBD might be interrupted, the higher value when interruption is improbable
In conclusion with a modern SSD there is very little reason to change the default especially because OS will use free RAM to opportunistically cache the filesystem in the free RAM anyway so a machine with higher RAM will always get an implicit speedup
"dbcache": {
"name": "DB Cache Size",
"description": "Set database cache size in megabytes; machines sync faster with a larger cache. Recommend setting as high as possible based upon machine's available RAM.",
"default": 450,
"min": 4
},Good point on flushing chainstate. This does make me want to re-run my benchmark sync with the default dbcache to see what the performance difference is.
I ran some tests and published my findings here: https://blog.lopp.net/effects-dbcache-size-bitcoin-node-sync-speed/