problem about the python Memory leak?
wangzelong0663 opened this issue · 2 comments
When I run the code below:
from kaggle_environments import evaluate
import gc
import objgraph
# Same definitions as "make" above.
environment = "halite"
configuration = {"episodeSteps":21, "size": 21, "randomSeed": 123456}
steps = []
# Which agents to run repeatedly. Same as env.run(agents)
agents = ["random", "random"]
# How many times to run them.
num_episodes = 1
count = 0
while 1:
rewards = evaluate(environment, agents, configuration, steps, num_episodes)
objgraph.show_growth()
print('count %s, objgraph.by_type:'%count,len(objgraph.by_type('list')))
count += 1And output logs are :
Loading environment football failed: No module named 'gfootball'
function 9630 +9630
dict 5061 +5061
tuple 4441 +4441
weakref 2372 +2372
wrapper_descriptor 2149 +2149
list 1500 +1500
method_descriptor 1387 +1387
getset_descriptor 1349 +1349
builtin_function_or_method 1190 +1190
type 1099 +1099
count 0, objgraph.by_type: 1500
list 1522 +22
Struct 49 +5
weakref 2373 +1
count 1, objgraph.by_type: 1522
list 1544 +22
count 2, objgraph.by_type: 1544
list 1566 +22
count 3, objgraph.by_type: 1566
list 1588 +22
count 4, objgraph.by_type: 1588
list 1610 +22
count 5, objgraph.by_type: 1610
list 1632 +22
count 6, objgraph.by_type: 1632
list 1654 +22
count 7, objgraph.by_type: 1654
list 1676 +22
...The len(objgraph.by_type('list') will add 22 same to the "episodeSteps":21 plus one all the time.
Even if I use the gc.collect().
Is this the problem about the python Memory leak? Is there anyway to fix it?
Because I find that when I run the evaluate function repeatedly for the very long time(such as with the loop of whlie True:), there are some problems with my memory.
I believe I fixed this issue in PR #157, can you confirm this? You can confirm this by checking out this branch, and then running your test.
I tried running your code with the main branch (leaking) and my branch (not leaking).
The leak is likely caused by logs and other function parameters that were passed as reference at function definition time.