khipu-io/khipu

Receipts storage optimization

dcaoyuan opened this issue · 2 comments

Firstly, it does not store information like Transaction hashes and block hashes with every single log entry. Secondly, it does not store bloom filter (256 bytes) with every transaction, but calculates it on the fly from the logs during the RPC request.

See Turbo-Geth

Bloom filter is 256 bytes, that is 256 * 6,000,000blocks * (tx per block) = 1.43G * (tx per block).

Assume tx per block is (600,000 tx per day) / (5760 blocks per day) = 100

The size in bytes of 6,000,000 blocks would be 1.43G * 100 = 143G.

Of course, the (tx per block) is less than 100 before i.e. 4,000,000 block.

After (at block 6285787)

23G	.khipu.eth/leveldb
457M	.khipu.eth/kesque.logs/receipts_idx-0
654M	.khipu.eth/kesque.logs/evmcode-0
655M	.khipu.eth/kesque.logs/td-0
50G	.khipu.eth/kesque.logs/body-0
3.8G	.khipu.eth/kesque.logs/header-0
13G	.khipu.eth/kesque.logs/account-0
457M	.khipu.eth/kesque.logs/td_idx-0
25G	.khipu.eth/kesque.logs/storage-0
38G	.khipu.eth/kesque.logs/receipts-0
4.4G	.khipu.eth/kesque.logs/account_idx-0
12G	.khipu.eth/kesque.logs/storage_idx-0
457M	.khipu.eth/kesque.logs/body_idx-0
262M	.khipu.eth/kesque.logs/evmcode_idx-0
457M	.khipu.eth/kesque.logs/header_idx-0
147G	.khipu.eth/kesque.logs
4.0K	.khipu.eth/keystore
169G	.khipu.eth/