linkedin/databus

How to capture changes from specific tables?

rahuljoshi123 opened this issue · 5 comments

rahuljoshi123 commented

I have specified mysql tables(LogicalSources) I want to capture changes from in sources.json .
When databus is getting an event for a table which is not specified in logical sources, I am getting null pointer exception in ORListener as I have not added schemas for these tables and have not configured these logical sources.
These are the exact lines -
VersionedSchema vs = _schemaRegistryService.fetchLatestVersionedSchemaBySourceName(_tableUriToSrcNameMap.get(tableName));
Schema schema = vs.getSchema(); // Here NullPointer

The application is running fine after this exception. Major concern here is, it is updating scn as -1 after handling non specified table events. So if the application stops after parsing events for unspecified tables, in the next run it will start from SCN -1.

Is there any good way to register specific tables and ignore others?

chavdar commented
rahuljoshi123 commented

Hi,
Thanks for the reply. Please let me know once you merge the fix.
Meanwhile if you share the fix branch, I can help in testing it.

Thanks,
Rahul

chavdar commented
rahuljoshi123 commented

Hi,

Fix you did is avoiding framing of avro records and the earlier exception of 'schema not available'.

But SCN is updated as -1 after processing a ignored source events. If relay restart after this, it will try to read events from scn -1.

I have done a fix - #85.
Please let me know your thoughts on this.

chavdar commented