np.array leaks memory

[rstarkov]

(edited to reflect latest findings)

This simple program beahaves as if it leaks memory:

    var arr = new float[10000 * 10000];
    while (true)
    {
        var arr = np.array(arr);
        arr.Dispose();
    }

The memory actually gets freed eventually, but not soon enough to avoid OOM exceptions under some conditions; further details in this comment.

[henon]

Can you support this claim with more data?

[rstarkov]

My bad, pressed Ctrl+Enter instead of Enter and apparently GitHub just posts the issue right away...

[rstarkov]

The reason I suspect that line is that if I replicate the same wrapping with np.zeroes it also leaks:

while (true)
{
    var leak = new NDarray<float>(np.array(np.zeros(new Shape(10000, 10000), np.float32)));
    leak.Dispose();
}

[henon]

so without wrapping in new NDarray( ) it doesn't?

[rstarkov]

It does if I leave np.array in that call.

Still leaky: using var x = np.array(np.zeros(new Shape(10000, 10000), np.float32));

Not leaky: using var x = new NDarray<float>(np.zeros(new Shape(10000, 10000), np.float32));

Can't say I follow the insides of np.array to understand why that line calls np.array on the ndarray, but it's that overload of np.array which seems to be leaking, perhaps?

[rstarkov]

@henon so perhaps it isn't an outright leak in that it's still possible for it to be deallocated. But it's extremely reluctant to deallocate np.array memory. Let me show you what I mean.

This here actually runs forever; it does not run out of memory, so technically there is no leak:

var arr = new float[1_000_000];
while (true) {
    var nparr = np.array(arr, np.float32);
    nparr.Dispose();
}

However, its peak usage can get absolutely massive, I've seen it as high as 22 GB - that's 22 thousand instances of this array that didn't get cleaned up.

This might have been OK if the RAM got freed under memory pressure, but unfortunately it doesn't always happen. Here's the same thing, but with a huge data set for Keras.NET:

var arr = new float[2_000_000_000];
while (true) {
    var nparr = np.array(arr, np.float32);
    nparr.Dispose();
}

In theory this isn't supposed to run out of memory, but on my system it only manages 9 iterations, and dies at 75 GB RAM usage with a PythonException: MemoryError.

Just to compare again to np.zeros, which is well behaved, the following code runs forever just fine, allocating and deallocating ~8 GB each time:

while (true) {
    var nparr = np.zeros(new Shape(2_000_000_000), np.float32);
    nparr.Dispose();
}

It never spikes, it always allocates and deallocates exactly when it's supposed to.

Can I do something on my end to force the np.array RAM to be freed on Dispose?

[henon]

Thanks for working on this. I wonder if the memory that doesn't get collected in time is on the .NET side or on the Python side. You can try forcing .net GC in that loop. If the problem goes away, it means the memory buildup is in .NET and we may be able to do something about it.

[rstarkov]

I have tried forcing a GC on both sides and it doesn't help much. I'm going to experiment with the Numpy.NET source next.

[rstarkov]

The "leak" comes from accessing ctypes.data to get a pointer to the np.empty array created inside np.array. Lots of hits on how to deal with this, none that look truly suitable. I'll keep looking but posting this in case someone knows the proper way to address that.

[rstarkov]

I've pushed a fix for the other path through np.array which had the same issue.

This one's worrying because it required me to dispose of the args tuple too. I'm guessing the library relies heavily on the GC disposing of these temporary values, which it does eventually but not reliably enough to be able to operate with huge arrays. So there might be other places that need the same treatment.

[henon]

Yes, but since most of the library's code is generated we can easily add disposing where needed in most parts. Only a few functions like np.array are hand-crafted. You can see this in the structure of the source files. I divided the manual stuff from the generated files.

[henon]

@rstarkov I just released your fix as https://www.nuget.org/packages/Numpy/3.11.1.31