Find better mechanism for syncing
Closed this issue · 1 comments
ksafranski commented
Per this comment by @TomFrost - using rsync
is going to have scans and syncs running full-throttle which is resource intensive. Need to find a solution which doesn't cause such heavy utilization and more elegantly syncs back to the host-mounted volume.
ksafranski commented
Current Plan:
- Slow down
rsync
ops to 30-60m cycles to prevent heavy runs and CPU spikes. - Have sync operations generate a "lockfile" to prevent multiple, concurrent runs, release on complete and check timestamp on existing to handle next-run checks
- Add a termination trap task on the container to ensure when container is exited it runs a final sync.
- (Maybe) add
--bwlimit
to rsync to throttle