Memory leak after upgrade to 1.38.0
vad opened this issue · 8 comments
Hi, we're experiencing memory issues after migrating from pushpin 1.37.0 to 1.38.0. The only other change we had was removing zurl from the configuration, pushpin was not starting otherwise.
This is the memory usage of the development setup, but we see the same pattern in production. You can clearly see when we upgraded to latest pushpin version. Dotted lines are k8s memory request/limit, we had to increase them.
Apparently memory increase is not determined by traffic: we have low traffic during the night and over the weekend, and very low traffic in the development setup.
Thanks for the report. Will look into this and report back.
Regarding zurl: is it possible you didn't have zurl installed? It's deprecated, but it should have worked if you had it installed and was referenced by your configuration, so I'm wondering what happened there. Maybe you used our updated Dockerfile (which no longer installs zurl) but with your own pushpin.conf?
zurl: exactly. I think I took pushpin.conf from the 1.37 docker a few months ago in order to customize the prometheus_port option, and zurl was there.
BTW I'm aware the report is short and probably incomplete. Let me know if I can provide more information
From left to right:
- orange: pushpin 1.38.0
- blue: pushpin 1.37.0 with zurl
- red: pushpin 1.37.0 without zurl
1.39.0 docker image was posted. It includes a memory leak fix, which was hopefully this.
I just updated to pushpin 1.39.1 the development instance:
Baseline usage increased compared to 1.37.0 but that was expected (RIP zurl). Memory leak looks solved.
Thank you
apparently there still is a leak:
I confirm there's still a leak.