apibara/dna

Enable multiple sinks per table/collection by adding additional conditions on invalidation

Closed this issue · 1 comments

fracek commented

Is your feature request related to a problem? Please describe.

At the moment, it's not possible to write to the same table using multiple sinks. Insertion works fine, but during invalidation one indexer deletes the other indexer's data too.

Describe the solution you'd like

I believe the easiest solution is to add a new option with a list of additional conditions for data invalidation.

For example:

export const config = {
  sinkOptions: {
    invalidate: [
      { column: 'my_column', value: 'abcd' },
      { column: 'other_column', value: 'aaa' }
    ]
  }
}

Obviously the user script must return objects that have my_column and other_column as attributes, even tho this behavior won't be enforced for simplicity sake.

All fields in invalidate will be added as AND expressions to the invalidate query, the condition being <column>==<value>.

Describe alternatives you've considered

We can add a callback in the script file to return the conditions at runtime. I believe it adds additional complexity for no reason.

Additional context

Affects the postgres and mongo sinks.

fracek commented

This is now available.