toluaina/pgsync

Checkpoint file not found error

Opened this issue · 2 comments

PGSync version: 3.1.0

Postgres version: Postgres 15

Elasticsearch/OpenSearch version: 8.14.1

Redis version: 7.2.1

Python version: 3.9.6

Problem Description: I am trying to prepare a POC having one table in my database and want to sync with index in my local.
Below is my schema json.

[
{
"database": "test_db",
"index": "leads",
"nodes": {
"table": "leads",
"columns": [
"id",
"lead_title",
"assignee1",
"assignee2",
"assignee3",
"created_by"
]
}
}
]

Error Message (if any): When i am running bootstrap --config schema.json command. I am getting below logs

2024-07-05 12:55:43.008:INFO:pgsync.utils: Settings:
2024-07-05 12:55:43.008:INFO:pgsync.utils: Schema : schema.json
2024-07-05 12:55:43.008:INFO:pgsync.utils: -----------------------------------------------------------------
2024-07-05 12:55:43.008:INFO:pgsync.utils: Checkpoint:
2024-07-05 12:55:43.008:INFO:pgsync.utils: Path: /Users/deepak/Projects/PG-Sync
2024-07-05 12:55:43.008:INFO:pgsync.utils: Postgres:
2024-07-05 12:55:43.008:INFO:pgsync.utils: URL: postgresql+psycopg2://postgres:****@localhost/postgres
2024-07-05 12:55:43.008:INFO:pgsync.utils: Elasticsearch:
2024-07-05 12:55:43.008:INFO:pgsync.utils: URL: http://elastic:********************@localhost
2024-07-05 12:55:43.008:INFO:pgsync.utils: Redis:
2024-07-05 12:55:43.008:INFO:pgsync.utils: URL: redis://localhost:6379/0
2024-07-05 12:55:43.008:INFO:pgsync.utils: -----------------------------------------------------------------
2024-07-05 12:55:43.059:INFO:elastic_transport.transport: GET http://localhost:9200/ [status:200 duration:0.012s]

  • public.leads
    2024-07-05 12:55:43.177:INFO:elastic_transport.transport: HEAD http://localhost:9200/leads [status:200 duration:0.003s]
    2024-07-05 12:55:43.177:WARNING:pgsync.sync: Checkpoint file not found: ./.test_db_leads
    2024-07-05 12:55:43.177:INFO:pgsync.redisqueue: Deleting redis key: queue:test_db_leads
    2024-07-05 12:55:43.258:INFO:main: Bootstrap: test_db

image



Hi @deepakbhatt-busy,
It is just a warning, you can ignore it.

Hi @Pavangj959 . Thanks for your reply.
I tried to run the 2nd command after running the 1st command (i.e., pgsync --config schema.json), but now I'm getting the following error
2024-07-08 10:22:12.112:ERROR:pgsync.base: (psycopg2.errors.ConfigurationLimitExceeded) all replication slots are in use
HINT: Free one or increase max_replication_slots. Please find attached screenshot for reference
.

image

I believe this is happening because when we executed the 1st command, it created a replication slot, and as mentioned in the documentation, my database configuration has max_replication_slots = 1.

and if i am deleting existing replication slot and running 2nd command again it is giving error.

RuntimeError: Replication slot "test_db_leads" does not exist.
Make sure you have run the "bootstrap" command.

image