toluaina/pgsync

Plugin don't work with delete or truncate action.

laeis opened this issue · 1 comments

laeis commented

PGSync version:
2.3.3
Postgres version:
13.6
Elasticsearch version:
8.7.1
Redis version:
latest
Python version:
3.7
Problem Description:
I created a plugin to filter documents that should not be synced. I use it when moving data from one shard to another because they are both connected to the same opensearch. It works well with Insert and Update. But when I delete the row in postgres, it seems that the plugin doesn't work in this case and also delete document in opensearch.
`
from pgsync import plugin
class ReadOnlyFilterPlugin(plugin.Plugin):
name: str = 'ReadOnlyFilter'

def transform(self, doc: dict, **kwargs) -> dict:
    if 'shard_mapping' in doc and doc['shard_mapping']['read_only'] == False:
        return doc
    else:
        return None

`

Error Message (if any):




from pgsync import plugin

class ReadOnlyFilterPlugin(plugin.Plugin):
    name: str = 'ReadOnlyFilter'

    def transform(self, doc: dict, **kwargs) -> dict:
        # Check if it's a delete operation
        if kwargs.get('operation') == 'delete':
            # Implement your logic for delete operation
            # For instance, return None to avoid deleting from Elasticsearch
            return None

        # For non-delete operations, continue with existing logic
        if 'shard_mapping' in doc and doc['shard_mapping']['read_only'] == False:
            return doc
        else:
            return None