Job logging
phockett opened this issue · 2 comments
phockett commented
Currently set in .json file per job and batch (directory).
Plan to process these globally and put in a DB for better job tracking.
phockett commented
Also: fix file naming scheme to avoid overwrite on rerunning jobs for additional E points (or add to existing log file).
phockett commented
To consider:
- with JSON/NoSQL route, should also push out consolidated data to DB (or, at least, small files that can be referenced/pulled from Zenodo to another website).
- DB for running jobs, as well as for processed jobs (currently only the latter).
- MongoDb for implementation?
- Django for web front-end (for ePSdata)?