aws/amazon-mwaa-docker-images

Generate per-subprocess CPU and memory metrics

rafidka opened this issue · 0 comments

Overview

In the scheduler container, we host multiple Airflow subprocesses, namely the scheduler, the DAG processor, and the triggerer. We need to generate per-subprocess metrics so we know when/if some processeses start others.

Acceptance Criteria

  • For each sub-process, generate CPU and memory metrics.
  • This should be as automated as possible, so any time we create multiple sub-processes in a single container, it is easy or no-effort to generate metrics for all sub-processes.

Additional Info

Context on this: #59 (comment)