Store cached function data against ABI & address (as a hash) rather than contract instance address only
tpmccallum opened this issue · 6 comments
Some contract instances will have multiple ABIs i.e. contracts with their actual ABI and then internal ABI association (a smaller ERC20 ABI inside a larger contract). When indexing these contract instances with nested ABIs we need a way to map the contract data (the result of calling all of the pure/view functions) with the contract instance. This can be achieved by creating a unique key (a hash of the contract instance address and also the appropriate ABI)
Actually, seeings how the function data of a particular contact instance is stored inside the index using the contract instance address, we can uniquely store the data in relation to a given associated ABI using just the ABI's sha3Hash.
The indexer harvester can construct the following format for the data
{
"functionData": {
"sha3unique0x123": {
"name": "the name",
"number": 1000,
"etc": "asdf"
},
"sha3unique0x456": {
"name": "the other name",
"number": 1111,
"etc": "asdf asdf"
}
}
}
The frontend can just fetch the data which relates to a particular known ABI hash (in cases where the search engine frontend is displaying a particular known smart contract).
In cases where the frontend does not know the ABI hash, the frontend can quickly de-duplicate the data.
For example, as you can see below, the BAT token's full ABI produces the following data, whereas, as you can also see below, the more simplistic (associated) ERC20 ABI produces a smaller set of function data (which is essentially redundant and can be deduped in the frontend using any language)
{'batFundDeposit': '0x88E2eFac3D2ef957FcD82Ec201a506871AD06204', 'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'batFund': '500000000000000000000000000', 'decimals': 18, 'tokenExchangeRate': 6400, 'version': '1.0', 'tokenCreationCap': '1500000000000000000000000000', 'isFinalized': True, 'fundingEndBlock': 3963480, 'symbol': 'BAT', 'ethFundDeposit': '0xAc2fa512DB158F44F5EE2FA5766ea7c282763cdB', 'tokenCreationMin': '675000000000000000000000000', 'fundingStartBlock': 3798640}
{'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'decimals': 18, 'symbol': 'BAT'}
In fact the Python (io.py file) that performs the call to the index can have an extra function added, which returns a distinct set of values (no redundancy). This will take a log of the work away from the frontend developer and the frontend display software i.e. JS.
Given the following redundant data examples, from the ERC20 compliant BAT smart contract instance:
Full BAT ABI
{'batFundDeposit': '0x88E2eFac3D2ef957FcD82Ec201a506871AD06204', 'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'batFund': '500000000000000000000000000', 'decimals': 18, 'tokenExchangeRate': 6400, 'version': '1.0', 'tokenCreationCap': '1500000000000000000000000000', 'isFinalized': True, 'fundingEndBlock': 3963480, 'symbol': 'BAT', 'ethFundDeposit': '0xAc2fa512DB158F44F5EE2FA5766ea7c282763cdB', 'tokenCreationMin': '675000000000000000000000000', 'fundingStartBlock': 3798640}
ERC20 ABI
{'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'decimals': 18, 'symbol': 'BAT'}
We can perform the following:
resultsList = [{'batFundDeposit': '0x88E2eFac3D2ef957FcD82Ec201a506871AD06204', 'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'batFund': '500000000000000000000000000', 'decimals': 18, 'tokenExchangeRate': 6400, 'version': '1.0', 'tokenCreationCap': '1500000000000000000000000000', 'isFinalized': True, 'fundingEndBlock': 3963480, 'symbol': 'BAT', 'ethFundDeposit': '0xAc2fa512DB158F44F5EE2FA5766ea7c282763cdB', 'tokenCreationMin': '675000000000000000000000000', 'fundingStartBlock': 3798640}, {'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'decimals': 18, 'symbol': 'BAT'}]
uniqueDict = {}
for result in resultsList:
for k, v in result.items():
if k in uniqueDict:
print("Already have " + k)
else:
uniqueDict[k] = v
print(uniqueDict)Which outputs the following clean and unique data
{'batFundDeposit': '0x88E2eFac3D2ef957FcD82Ec201a506871AD06204', 'name': 'Basic Attention Token', 'totalSupply': '1500000000000000000000000000', 'batFund': '500000000000000000000000000', 'decimals': 18, 'tokenExchangeRate': 6400, 'version': '1.0', 'tokenCreationCap': '1500000000000000000000000000', 'isFinalized': True, 'fundingEndBlock': 3963480, 'symbol': 'BAT', 'ethFundDeposit': '0xAc2fa512DB158F44F5EE2FA5766ea7c282763cdB', 'tokenCreationMin': '675000000000000000000000000', 'fundingStartBlock': 3798640}
The main harvester now produces the following format for functionData
"functionDataList": [
{
"abiHash": "0xf184e89595256b4eff3b1f0a66570fb6944e04eab99156d3f7bfe5b7c082c628",
"functionDataId": "0xc302313119f8116adcdf297f592f7709c47a0684608ade8539cdc3e5fbb4ce4b",
"functionData": {
"batFundDeposit": "0x88E2eFac3D2ef957FcD82Ec201a506871AD06204",
"name": "Basic Attention Token",
"totalSupply": "1500000000000000000000000000",
"batFund": "500000000000000000000000000",
"decimals": 18,
"tokenExchangeRate": 6400,
"version": "1.0",
"tokenCreationCap": "1500000000000000000000000000",
"isFinalized": true,
"fundingEndBlock": 3963480,
"symbol": "BAT",
"ethFundDeposit": "0xAc2fa512DB158F44F5EE2FA5766ea7c282763cdB",
"tokenCreationMin": "675000000000000000000000000",
"fundingStartBlock": 3798640
}
}
]
As you can see each functionData item in the list corresponds to a particular ABI abiHash and also has its own unique functionData and the hash of that data (used for efficient update checking) called functionDataId.