MongoEngine/mongoengine

`EmbeddedDocumentListField` causes additional memory usage.

neolooong opened this issue · 0 comments

This is the minimal example I found:

class Comment(EmbeddedDocument):
    content = StringField()

class Post(Document):
    title = StringField()
    comments = EmbeddedDocumentListField(Comment)

if Post.objects.count() == 0:
    for i in range(100):
        Post(title=f"{i}", comments=[Comment[f"{i}"]]).save()

def ok():
    qs = Post.objects
    for post in qs:
        post.title

def bug():
    qs = Post.objects
    for post in qs:
        post.comments

gc.collect()
ok()
print("collect after ok():", gc.collect())  # 0
bug()
print("collect after bug():", gc.collect())  # collect many objects

gc.collect() does not collect anything after exiting ok(). However, it can collect some objects after exiting bug().
The more times Post.comments accessed, the more memory used.
Although this may not a big problem since it can eventually collect internally by python.
In my case, this increase several gigabytes of RAM within a few minutes.