Provide simultaneous/parallel updating of all search engine data

[tpmccallum]

The idea is to allow all ABIs in the config to be processing in parallel.
Further, for each of the contract instances (which are an instantiation of the ABIs master contract) are to be processed in parallel.
Even further, each of the subsets of data inside a contract instance (i.e. one of the individual fields such as "player_addrs" etc) are to be processed in parallel.

[tpmccallum]

We now use multithreading and queuing to achieve this.
The amount of threads is governed by the amount of items being indexed.
For example, the number of threads to kick off harvesting of each ABI is set to the total amount of ABIs which need to be harvested i.e. we do everything at one time (in parallel)

self.threadsupdateStateDriverPre = []
for i in range(len(self.abis)):
            tupdateStateDriverPre = threading.Thread(target=self.updateStateDriver, args=[queueIndex])
            tupdateStateDriverPre.daemon = True
            tupdateStateDriverPre.start()
            self.threadsupdateStateDriverPre.append(tupdateStateDriverPre)

Also the amount of parallel threads (in relation to indexing each of the contract instances) is set to the number of contract instances (dynamically as shown above for the ABIs)

self.threads = []
threadCount = len(self.uniqueContractList)
            for i in range(threadCount):
                t = threading.Thread(target=self.worker, args=[_esIndex, _contractAbiJSONData, queueIndex])
                t.start()
                self.threads.append(t)

[tpmccallum]

Real time - At this stage everything is processed once in under 10 seconds. This provides a real-time equivalent (in the event that the block time is around 10 seconds).

[tpmccallum]

If the code finishes in under 10 seconds then it will rest (to make up the difference). There is no need to process the data again if a block has not been produced (this will just be redundant work).

If for some reason the code takes longer than 10 seconds to finish, then the code will run again straight away, without delay.

#Start of function
self.upcomingCallTimeState = time.time()
# Code to process goes here ...

# After processing ...
# set the time interval for when this task will be repeated
            self.upcomingCallTimeState = self.upcomingCallTimeState + 10
            # If this takes longer than the break time, then just continue straight away
            if self.upcomingCallTimeState > time.time():
                time.sleep(self.upcomingCallTimeState - time.time())