second-state/smart-contract-search-engine

Allow object to be passed into harvest.py (full, topup and state)

tpmccallum opened this issue · 1 comments

The harvest.py script will eventually access the "all" index. The "all" index which is created by the harvest_all.py script is a master list of every contract (regardless of whether an ABI is present). Users can upload ABIs to the "all" index via an interface (which will validate the ABI safely). Once the ABI is available for a particular contract instance, the search engine can call all of the public functions and create a rich index of the state of the contract. This not mean that the search engine has to process every contract though. In the cases where a particular instance of the search engine just needs to harvest 1/2 dozen contracts it can still query the "all" index for the authoritative information such as contract address, ABI, Bytecode etc. but the search engines query does not have to return everything; just what it wants. This can be achieved by passing in a query object like this

{"query": {"match": {"contractAddress": "0x0fEB15a0F7029b0F4aF355aa2Fc3CFa6Df7C8483"}}}'

This no longer needs to be done.
The harvest.py will index all instances of a contract if that contract’s ABI is in the “abi” index.
Further, the harvest.py creates and indexes (in the common index) a Sha3 hash of the ABI in question. What this means is that any contract instance that is indexed in the common index is able to find its full ABI in the “abi” index (because the full ABI is stored in the “abi” index using its Sha3 hash as its _id)
Just for clarity, the purpose of the harvest_all.py is to index (in the “all” index) every contract instance regardless of whether an ABI is known, or if Bytecode is known. The sole purpose of the “all” index is to allow users to see an index of every contract (search their own contracts by, say, address or transaction has etc). Web interfaces like the upload_abi.html will access the “all” index and facilitate the uploading of ABIs (into the “abi” index) by providing a mechanism to associate addresses with ABIs (and/or addresses with Bytecode; which are similarly stored in the “bytecode” index with the Sha3 of the Bytecode as its _id).
The harvest.py is the able to harvest contract state and contract event logs by processing every block whilst utilising the known ABI and Bytecode (facilitating web3 contract instance creation to do all of this).