External transaction index for Bitcoin Core

Status

This project is still experimental, use at your own risk!

How does it work

The indexer connects to a fullnode over the p2p protocol (at the moment 127.0.0.1:8333 mainnet) and performs the following actions:

opens database
connect to the peer
sync all headers, find unindexed blocks
requests blocks that are not yet indexed
index all found transactions txid->blockhash
once all blocks (found via header) has been requested, idle for new inv/block messaged

Be aware!

The indexer trusts the remote peer! Only connect to a trusted peer, ideally via localhost. If you connect via the public internet, make sure you connect through a secure channel.

Database

The indexer currently works only with leveldb (there was a LMDB implementation, check git history).
In order to save space, each block will get an internal blockmap-key (uint32 / 4bytes). That internal blockmap-key must not be confused with the block-height. The blockmap-key may be different on other systems/instances.

Space requirements

~14.3GB up to block 522081 (May 10th 2018)
complete index up to 522081 takes about ~60min on a fast CPU with SSD

REST interface

Lookup can be done via HTTP 18445
Two endpoints ** /blockhash/<txhash> will return the blockhash of the requested transaction id ** /tx/<txhash> will return the transaction of the requested txid (will fetch the relevant block via p2p and return the tx)

TODO:

Reduce memory usage with internal header maps
Lookup should also respect the non-flushed cache
Make sure that incoming block invs during sync-via-headers do not confuse the indexer
Add file based logging
Add runtime option to switch network (testnet, regtest) and the IP to connect to

Build

./autogen.sh
./configure
make -j

run

./src/bitcoincore-indexd

Options

./src/bitcoincore-indexd --datadir=/tmp (change datadir, default is ~/.bitcoincore-indexd)
./src/bitcoincore-indexd --dbcache=xxx (change the leveldb dbcache)

fotispap/bitcoincore-indexd