memory leak in publish_to_topic()

[bencaby]

Describe the bug

The memory of the process monotonically increases when publishing to ipc topics.
This is not happening at each call of the publish_to_ip() method but very x call the process gain about 3.9kB memory usage.

Expected Behavior

memory usage should be stable

Current Behavior

memory usage is increasing

Reproduction Steps

import json

from awsiot.greengrasscoreipc.clientv2 import GreengrassCoreIPCClientV2
from awsiot.greengrasscoreipc.model import PublishMessage, BinaryMessage
from memory_profiler import profile
from time import sleep

def main():
    ipc_client = GreengrassCoreIPCClientV2()
    count = 0
    while True:
        payload = dict(value=count)
        count += 1
        publish(payload, ipc_client)


@profile(precision=4)
def publish(payload, client):
    message = PublishMessage(binary_message=BinaryMessage(message=json.dumps(payload).encode()))
    client.publish_to_topic(topic="readings/test/test_mem", publish_message=message)
    sleep(0.1) # to avoid 100% cpu
    return


if __name__ == "__main__":
    main()

Possible Solution

No response

Additional Information/Context

No response

SDK version used

awsiotsdk 1.12.5

Environment details (OS name and version, etc.)

Linux Raspberry pi OS buster

[MikeDombo]

You seem to be using memory_profiler, what does that tell you is the cause of the leak?

One thing to see is if the leak is in Python or native code. Run your loop many times (1,000 or 10,000, something like that), then force a garbage collection and finally print the objects that are still alive.

import gc
gc.collect()
print(gc.get_objects())

If there are a large number of objects still alive after garbage collection, then there may be a leak in the Python code somewhere. If there is a smaller number (and not growing over time), then there is no leak in the Python code.

[bencaby]

@MikeDombo I added the gc code but I print len(gc.get_objects()) each 1000 publish.
in this case I get a stable number of objects

Remaining Objects: 23906
Remaining Objects: 23905
Remaining Objects: 23905
Remaining Objects: 23905
Remaining Objects: 23905
Remaining Objects: 23905
Remaining Objects: 23905

the line that produce a delta memory (using memory_profiler) seems to be line 624 of awsiot/eventsreamrpc.py
return self._continuation.activate(
operation=self._model_name(),
headers=headers,
payload=payload,
message_type=protocol.MessageType.APPLICATION_MESSAGE)

[bencaby]

Hi any news on this issue ?
I updated to awsiotsdk 1.15.4 (with awscrt-0.16.21) and the issue is still there

[yasminetalby]

Hello @bencaby ,

My apologies for the delay. We are currently working on investigating the root-cause of this issue. I will post a comment as soon as an update is available.

Thank you very much for your feedback and contribution.
Sincerely,

Yasmine

[bencaby]

Hi, a new element to help you to find the root cause.
It seems that there is no memory leak using python 3.9.2 but well using python 3.11.0
it may be a python related bug