Investigate bitcoind prune mode for payment validation

[lemonlatte]

In order to validate transactions in an efficient way, we decide to create a way to validate bitcoin payment faster. This part is to first to benchmark the resource usage for bitcoind.

[lemonlatte]

Current mechanism

Here are two different mechanism now we can validation a payment for bitmark transactions:

1. REST Interface from bitcoind and litecoind
2. Use discovery service

No matter which one we use, we need to run full nodes for both Bitcoin and Litecoin.

Bitcoin / Litecoin Rest Interface

Bitcoind and litecoind provide a series of REST api for users to query block and transaction data. Due to the security concern, it is suggested to run the bitmarkd, bitcoind and litecoind in the same node, without exposed publicly. That causes the scale of bitmarkd nodes painful because the cost of running those nodes is huge.

Another thing is that its RPC is not efficient. It would become a bottleneck if we connect too many bitmarkd node to them for querying payment transactions.

Node Analysis

Full node storage usage

• Bitcoin mainnet - 267G
• Litecoin mainnet - 26G
• Bitcoin testnet - NO DATA
• Litecoin testnet - 2.4G

Sync time of first launch

• When we use m5.large instance on AWS, it takes about 21 hours of sync time for running a bitcoind node on mainnet.

Prune mode

• Storage size can be determined by configuration. For example, -prune=1024 keeps the latest 1024MB of block data.

• If prune mode is enabled, bitcoind and litecoind can still offer correct transaction data only if the referenced block data is not pruned.

  • On Litecoin testnet, we can fetch blocks back to one day before when we set -prune=550 (550MB block data left).
  • For data which is not available (pruned), it returns 119a4610f6b11225a580f013f2be12523bd9733b3f014d24e71dfeb8abe0edd8 not available (pruned data)

The prune mode actually saves lots of spaces. I think it is the only advantage of it. But it requires setting a proper size to provide a suitable payment validation capability. The number will also be affected by the network activities.

The fact that users need to create its bitcoind / litecoind node instead of leverage the current network makes it hard for users runing bitmarkd node.

A better solution would be integrate the bitcoin SPV protocol into bitmarkd and make it be able to leverage the bitcoin network for validating transaction payments.