toluaina/pgsync

DELETE not working

carlos-pereira-costoya opened this issue · 3 comments

carlos-pereira-costoya commented

PGSync version: 3.0.0

Postgres version: 16

Elasticsearch/OpenSearch version: 1.2.0 (OpenSearch

Redis version: 7.2

Python version: 3.9

Problem Description:

Hi, we are running local tests with the example docker-compose from toluana GitHub repository, using both non-partitioned and partitioned tables in PostgreSQL. When performing INSERTs and UPDATEs, the PGSync service detects the changes perfectly. However, when executing DELETEs, it is not reflecting the changes in OpenSearch. The DETACH command is also not working on partitioned tables.

Interestingly, deleting the index in OpenSearch and rerunning PgSync successfully syncs the deleted records to OpenSearch. Is this a known bug, or is there a missing parameter in the configuration?

Error Message (if any):
toluaina commented

Are you able to test this with OpenSearch 2.11.1?
Can you also confirm that you ran bootstrap successfully?
are there any errors in the terminal?
finally, can you run off the latest master branch and confirm is this happens?

carlos-pereira-costoya commented

Hi @toluaina ,

I retested in OpenSearch 2.11, performed a complete load, and cleaned the replication slots in PostgreSQL.

The issue was that I had replication slots with old tests in the Docker environment, and in the cloud environment, I didn't have any volume available to create the checkpoint files. In these cases, the pgsync synchronization wasn't working properly and the delete was nott working.

Thanks for your help, it's much appreciated.

carlos-pereira-costoya commented

Solved, I close the issue