Allow taskgraph to optionally log a heartbeat
phargogh opened this issue · 2 comments
phargogh commented
Sometimes with long-running processes, it's nice to have a heartbeat of the overall progress of tasks. The NDR+ pipeline for NCI does a nice job of this, even though it isn't using taskgraph for the actual monitoring of model runs.
Things that would be nice to include in this heartbeat might be to log the following every n
seconds:
- number of tasks in the tree
- number of tasks completed
- number of tasks remaining
- (maybe) average runtime of tasks
dcdenu4 commented
I just remembered the reporting_interval
argument that can be passed in to TaskGraph. It logs every n
seconds with a lot of that information listed above.
phargogh commented
Sure does! Closing.