ADD submission pipeline log capture
wabscale opened this issue · 0 comments
We'll want to start off with adding a new pipeline_log
blob column to the submission table
Anubis/api/anubis/models/__init__.py
Line 484 in a5e3f00
Create the migration for it (running it will be tricky as it will definitely take a minute to run)
Then replace this with something that will get the logs and store them on some new column of the submission:
Anubis/api/anubis/k8s/pipeline.py
Lines 153 to 156 in a5e3f00
I'm not sure we really need to be explicitly deleting jobs as they finish. We can just rely on kubernetes cleaning them up, or we can add the job ttl https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/