vitabaks/autobase

WAL Archiving Occurs Every Second Despite archive_timeout Set to 600s

Closed this issue · 3 comments

I am experiencing an issue with PostgreSQL where WAL files are being archived every second, even though the archive_timeout parameter is configured to 600 seconds(10min). This behavior is causing an unexpected frequency of WAL archiving, which might lead to performance degradation and excessive storage usage.

Configuration Details:

PostgreSQL Version: 17.2
WAL Segment Size: 16MB
min_wal_size: 2GB
max_wal_size: 16GB
wal_buffers: 32MB

Relevant Parameters:

archive_timeout = 600s
wal_segment_size = 16MB
max_wal_size = 16GB
Archiving is configured using wal-g.

Observed Behavior:
WAL files are archived every second (of 1-3MB of each wal file to the s3).
This occurs regardless of the low volume of write operations in the database.
Checking pg_stat_archiver shows rapid increments in the archived_count.
Expected Behavior:

WAL files should only be archived either when the 16MB segment is full or when the archive_timeout of 600 seconds is reached.
archive_command uses WAL-G to push the WAL files to an S3-compatible storage.
#du -sm /var/lib/postgresql/17/main/pg_wal/
16385   /var/lib/postgresql/17/main/pg_wal/

Please provide guidance on whether this is a configuration issue or a potential bug. Let me know if additional details are required for further debugging.

WAL archiving in PostgreSQL occurs either when a WAL segment is filled (default size is 16 MB) or after the time specified in the archive_timeout parameter has elapsed. If archiving happens every second, it likely indicates a write workload of approximately 1 WAL per second.

This is not an issue and is normal behavior for a high-load database. In fact, I have setups where 3-4 WAL files are archived per second, and this is also considered standard system operation.

Check the metrics of the monitoring system, I recommend using pgwatch

Solved by run full backup once:
wal-g backup-push /var/lib/postgresql/17/main/