RT Archiver: Copy prod configs to test occasionally
Closed this issue · 3 comments
As a Cal-ITP GTFS data user, I want the test RT archiver not to report false-alarm Sentry issues because of outdated configurations so that I don't waste time troubleshooting something that isn't a real issue.
Today we ran into an issue where the RT archiver test instance was using a really old config file that contained deprecated URLs. We don't want to run a service that's hitting unused URLs. The challenge is that the test archiver looks in the test-calitp-gtfs-download-config bucket for its configs, and we don't regularly update the data in there.
We could consider setting up a ~weekly job to copy the latest prod config into the test bucket so the test archiver doesn't get too out of sync.
Example get latest import/usage: https://github.com/cal-itp/data-infra/blob/main/airflow/dags/download_gtfs_schedule_v2/download_schedule_feeds.py#L25
For testing -- check that the test archiver is picking up the new config file and is working as expected: https://monitoring.k8s.calitp.jarv.us/d/AqZT_PA4k/gtfs-rt-archiver?orgId=1&refresh=30s&var-namespace=gtfs-rt-v3-test