Docker Compose Example(s) Not Working
rileyhun opened this issue · 0 comments
rileyhun commented
PGSync version: 3.0.0
Postgres version: postgres:14
Elasticsearch/OpenSearch version: latest
Redis version: latest
Python version: 3.9.0
Problem Description:
I have been stuck getting pgsync
working for the past week and I've tried tinkering around w/ different configurations, but nothing seems to be working.
Error Message (if any):
pgsync | (Background on this error at: https://sqlalche.me/e/14/e3q8)
pgsync | Traceback (most recent call last):
pgsync | File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
pgsync | self.dialect.do_execute(
pgsync | File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
pgsync | cursor.execute(statement, parameters)
pgsync | psycopg2.errors.ConfigurationLimitExceeded: all replication slots are in use
pgsync | HINT: Free one or increase max_replication_slots.
Reproducible example:
conf.sql
ALTER SYSTEM SET wal_level = logical;
ALTER SYSTEM SET max_replication_slots = 1;
populate.sql
CREATE TABLE IF NOT EXISTS product (
pk_id SERIAL NOT NULL,
name varchar(100) NOT NULL,
PRIMARY KEY ("pk_id")
);
INSERT INTO product (name) VALUES ('test product1');
INSERT INTO product (name) VALUES ('test product2');
INSERT INTO product (name) VALUES ('test product3');
schema.json
[
{
"database": "postgres",
"index": "product",
"nodes": {
"table": "product",
"columns": []
}
}
]
PgsyncDockerfile
FROM python:3.9
ARG WORKDIR=/app
RUN mkdir $WORKDIR
WORKDIR $WORKDIR
COPY ./pgsync ./
RUN pip install git+https://github.com/toluaina/pgsync.git@3.0.0
RUN chmod +x ./wait-for-it.sh
RUN chmod +x ./runserver.sh
PostgresDockerfile
FROM postgres:14
COPY ./pgsync/conf.sql /docker-entrypoint-initdb.d/
COPY ./pgsync/populate.sql /docker-entrypoint-initdb.d/
RUN chmod a+r /docker-entrypoint-initdb.d/conf.sql
RUN chmod a+r /docker-entrypoint-initdb.d/populate.sql
docker-compose.yml
version: "3.8"
services:
opensearch:
image: opensearchproject/opensearch:latest
container_name: elasticsearch
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- "DISABLE_INSTALL_DEMO_CONFIG=true"
- "DISABLE_SECURITY_PLUGIN=true"
- ES_JAVA_OPTS=-Xms512m -Xmx512m
ulimits:
memlock:
soft: -1
hard: -1
ports:
- '9200:9200'
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:2.1.0
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
- OPENSEARCH_HOSTS=http://opensearch:9200
- DISABLE_SECURITY_DASHBOARDS_PLUGIN=true
db:
build:
context: .
dockerfile: PostgresDockerfile
container_name: database
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "5432:5432"
volumes:
- db:/var/lib/postgresql/data
- ./init-db.sql:/docker-entrypoint-initdb.d/init-db.sql
redis:
image: "redis:alpine"
container_name: redis
ports:
- "6379:6379"
pgsync:
build:
context: .
dockerfile: PgsyncDockerfile
container_name: pgsync
command: ./runserver.sh
sysctls:
- net.ipv4.tcp_keepalive_time=200
- net.ipv4.tcp_keepalive_intvl=200
- net.ipv4.tcp_keepalive_probes=5
labels:
org.label-schema.name: "pgsync"
org.label-schema.description: "Postgres to OpenSearch sync"
com.label-schema.service-type: "daemon"
depends_on:
- db
- redis
- opensearch
environment:
- PG_USER=postgres
- PG_PASSWORD=postgres
- PG_PORT=5432
- PG_HOST=db
- PG_DATABASE=postgres
- ELASTICSEARCH_PORT=9200
- ELASTICSEARCH_SCHEME=http
- ELASTICSEARCH_HOST=opensearch
- REDIS_HOST=redis
- REDIS_PORT=6379
- LOG_LEVEL=WARNING
- ELASTICSEARCH=false
- OPENSEARCH=true
restart: always
volumes:
db:
driver: local