jeffchan/blockchain-parser

Failed to open block-chain input file at blk00509.dat

Opened this issue · 5 comments

OPrtr commented

Hello Jeff,

In case you would be willing to revisit this script and help out.

Ok, so this is where it stops:
Scanned 410,224 block headers so far, 188 since last time.
Found 410,224 block headers total. 0 in the last block.
Finished scanning block headers. Built block-chain with 410202 blocks found..

Also it crashes somewhere around 320 000 blocks while processing statistics and I don't see it allocating more than 8Gb of memory at any point in the process.

Can you help? I could provide all the feedback you would need. After alot of research it seems that there is no longer a functional parser out there with the blockchain this big.

Sorry I’m not sure I can help :( I haven’t worked on this in quite awhile

OPrtr commented

Too bad, it seems to be the most efficient piece of code lying around. If you have a general idea as to where could I look please let know. It just crashes without any output.

Thanks anyway

The cause is that code opens a lot of file handles (for each *.dat file) and keeps them open. C standard library has limit of 512 opened file handles. To solve this, change MAX_BLOCK_FILES to 8192 (in Blockchain.cpp) and call _setmaxstdio(8192) at the beginning of main function.

Also change MAX_TOTAL_BLOCKS to 1000000 (in Blockchain.cpp) and call max_blocks 1000000 before calling scan command.

Note these values will change in future as blockchain grows. At the moment of this comment we have 707646 blocks.

Overall, after spending few hours updating the code to make it work - my conclusion is the code is beyond repair. Don't waste you time and move on.