DCGM_FI_DEV_GPU_UTIL Abnormal Output

[Jea-Eok-Kim]

Hello
You want to use the gpu monitoring system in grafana using dcgm-exporter and prometheus, but the value of gpu usage is strangely output.

Grafana screen output is as follows.

As you can see on the screen, GPU utilization is 9223372036854776000%.

The dcgmidmon -e 203 (DCGM_FI_DEV_GPU_UTIL) command results are as follows:

The MIG Enable status is as follows.

Please let me know if there is a way to obtain GPU usage rate even in MIG Enable.

Please help me.