Sync stops in some conditions

[joojis]

A worker can fail during sync(). There are various reasons that make the worker fail, but, currently, it's retrying only for the specific HTTP connection errors.

codechain-indexer/src/worker/index.ts

Lines 42 to 59 in 0d95df0

	public run = async () => {
	this.watchJob = scheduleJob(this.config.watchSchedule, async () => {
	try {
	if (this.lock.isBusy(ASYNC_LOCK_KEY) === false) {
	await this.sync();
	}
	} catch (err) {
	const error = err as Error;
	if (error.message.search(/ECONNRESET|ECONNREFUSED/) >= 0) {
	console.error("RPC Error");
	} else {
	console.error(error);
	this.watchJob.cancel(false);
	}
	}
	});
	this.watchJob.invoke();
	};

We need to change it to retry for more reasons like:

• DB timeout
• async-lock timeout
• etc.