louis-frayser/plan-c

Server crashes on startup due to faulty [duration] data in on-disk cached data.

Opened this issue · 1 comments

A disk file (actually 2 of them from over a month ago) contained '0:71' instead of '1:11' for the duration of a practice session.

The server promptly crashed on an exception thrown by the Postgres database.

Thu Jun 20 11:56:55 PDT 2024
Log file:
query-exec: interval field value out of range: "0:71"
SQLSTATE: 22015

On-disk cache that collects data while the RDBMS is down:
grep -r 71 .
./2024-05-12/assoc-13:21:04.scm:(("Music Practice" "Cello") . "0:71")
./2024-05-12/assoc-13:19:28.scm:(("Music Practice" "Cello") . "0:71")

More background:

The SQL server, Postgress, was recently recycled, and apparently it restarted with an old set of files. So, a related app, that depends on data from the past 7 days was also crashing, due to null, '(), being return from a query for recent data.

I manually fixed the 0:71:00 in the data files, then noticed that that data was actually the newest data in the store. Next, to find the missing data. I'll try the cron-based backups.