kubepack/swift

Connection leak?

Closed this issue · 5 comments

Nitrogenycs commented

I have swift and tiller running and make a request to swift every 3 seconds. If I let this run for like 30 minutes, the tiller and swift containers have their memory growing without bounds.
The reason for this seems to be that the number of connections between swift and tiller is continously growing.

Connection count already over 1800 and continously growing:

~ $ netstat | wc -l
1817
~ $ netstat | wc -l
1822
~ $ netstat | wc -l
1826

Excerpt from netstat connections (all target tiller:44134):

tcp        0      0 swift-7559bd6fcd-8qwwg:35734 tiller-deploy.kube-system.svc.cluster.local:44134 ESTABLISHED
tcp        0      0 swift-7559bd6fcd-8qwwg:58532 tiller-deploy.kube-system.svc.cluster.local:44134 ESTABLISHED
tcp        0      0 swift-7559bd6fcd-8qwwg:50310 tiller-deploy.kube-system.svc.cluster.local:44134 ESTABLISHED

I guess the connections are not properly closed?

tamalsaha commented

@Nitrogenycs, can you try the image appscode/swift:close and see if that fixes this connection leak problem. I can cut a patch release, if #68 fixes the issue.

Nitrogenycs commented

I can confirm that appscode/swift:close actually fixes the bug. There is no increase in memory anymore.

Thanks a lot for fixing this bug so quickly!

  • 🎉1
tamalsaha commented

@Nitrogenycs , I have published appscode/swift:0.5.1 with this fix. https://github.com/appscode/swift/releases/tag/0.5.1

Please upgrade to that.

naivenut commented

@tamalsaha Thank you for making a new release!

The swift chart at https://github.com/kubernetes/charts/tree/master/stable/swift is still at 0.5.0. For now I just used --set swift.tag=0.5.1 to use the new image.
Does one have to open a PR at the kubernetes chart repository every time there is a small change to any of the charts (e.g. just changing the image version)?

tamalsaha commented

@naivenut, I have submitted a new pr to update the stable charts. helm/charts#2872