Task data continues to be reported after a task is gone

[barkerd427]

I'm not sure why, but when I look at the Graph in Prometheus of mesos_task_cpus_system_time_secs or mesos_task_mem_rss_bytes I get static results from tasks that are long since gone. I think all mesos prefixed metrics do the same thing. What I mean by static is a solid horizontal line running across the entire graph as if the exporter is providing the last value for that task. When I look at the actual /metrics endpoint, I do see that all tasks are reported when I only have a couple running.

[antonlindstrom]

I've noticed this as well, I'm not really sure how to solve this. I'll try to take a look and see if I can somehow timeout tasks that no longer exists.

Thanks!

[barkerd427]

I'll try to look at the code also.

On Sun, Mar 1, 2015 at 2:36 AM, Anton Lindström notifications@github.com
wrote:

I've noticed this as well, I'm not really sure how to solve this. I'll try
to take a look and see if I can somehow timeout tasks that no longer exists.

Thanks!

—
Reply to this email directly or view it on GitHub
#2 (comment)
.

[barkerd427]

It looks like the endpoints return an empty array when there is nothing running on the slave, so we just need to remove the gauge perhaps when the task is no longer running.