CLI freeze if dump file is too big
sfairat15 opened this issue · 1 comments
sfairat15 commented
At current realisation, dumper write trace/dump content into stdout.
Then CLI read log from job pod and write it into local file.
If trace/dump file is too big, then Docker at k8s node rotate log files, CLI freezes, no files copied at the local FS.
See - https://docs.docker.com/config/containers/logging/json-file/
Sample configuration at k8s node:
{
"live-restore": true,
"log-driver": "json-file",
"log-opts": {
"max-size": "50m",
"max-file": "5"
}
}
Some suggestions:
- Send files into blob
- Send files into PVC
- Use
kubectl cp
mechanism to copy file from pod to local FS.
StupidScience commented
fixed by #6