Kong/kong-python-pdk

Plugin memory usage increasing when calling `kong.response.error`

abhisheksharma2805 opened this issue · 5 comments

Kong: 2.7.0
python-pdk: 0.30

Plugin code:

import kong_pdk.pdk.kong as kong

Schema = (
    {"name": {"type": "string"}},
)

version = '0.1.0'
priority = 1000


class Plugin(object):
    def __init__(self, config):
        self.config = config

    def access(self, kong: kong.kong):
        return kong.response.error(401, "Auth failed")

I kept hitting the endpoint with this plugin enabled in a loop for almost an hour.

At the start memory usage was ~300MB. This kept increasing continuously and about an hour later memory usage reached around 800MB. This will not stop and eventually memory usage reaches the pod limit after which the plugin server restarts.

Screenshot 2022-04-28 at 1 34 14 PM

Recorded the memory usage of the plugin via top command

At initial stages (~70-80MB)
Screenshot 2022-04-28 at 12 51 24 PM

After approx 1 hour (more than 500MB)
Screenshot 2022-04-28 at 1 39 46 PM

In production I have seen this memory even reach 4GB (pod memory limit) after which the plugin restarts (memory dropped back to 300MB)
Production graph (plugin was different than the above one but that is also returning kong.response.error for invalid requests):

Screenshot 2022-04-28 at 1 43 38 PM

Surely there is memory leak somewhere. Please resolve this ASAP, I am stuck on this for almost 2 months now.
There was a fix that happened in 0.30 but that was only able to remove ghost threads but memory usage is still the issue.

@fffonion
Do we have any timeline when this issue will be resolved?