Uploading files to GCS via WriteChannel cause the memory to increase - memory not released

[ihudedi]

Hi
I am uploading files to GCS bucket using WriteChannel
WriteChannel writeChannel = storage.writer(blobInfo);
writeChannel.write(byteBuffer);
writeChannel.close();

After many files being uploaded the memory increased and doesn't released.

Thanks,
Itay

[BenWhitehead]

Hi @ihudedi,

When creating a WriteChannel an internal buffer is created to perform chunking of the upload to GCS.

By default, the buffer size is 16MiB. Depending on which JVM you're using along with which garbage collector, the allocated buffers can sometimes be immediately placed into the Old Generation of the heap. Once in the Old Generation, they only way they are garbage collected is when a full collection cycle happens. If you're app is not generating enough garbage to cause the Old Generation to need a full collection often the JVM won't.

If the objects you are uploading are smaller than 16MiB, it probably makes sense to reduce that size by doing

try (WriteChannel writeChannel = storage.writer(info)) {
  writeChannel.setChunkSize(4 * 1024 * 1024); // 4MiB instead of the default of 16MiB
  writeChannel.write(byteBuffer);
}

If you're able, could you share which JVM, version and GC settings you're using?

[ihudedi]

Hi @BenWhitehead
java version "11.0.10" 2021-01-19 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.10+8-LTS-162)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.10+8-LTS-162, mixed mode)
-XX:+UseParallelGC -XX:-TieredCompilation -Xmx2048m -Xshare:off -XX:MaxDirectMemorySize=512m -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=10 -XX:+HeapDumpOnOutOfMemoryError

[BenWhitehead]

Can you try adding -Djdk.nio.maxCachedBufferSize=33554432 to your command line? I just learned that in java11 there is a jdk built in ByteBuffer cache which attempts to help, but since storage uses large buffers by java standards this could be causing the resident memory of the process to be inflated. Setting this system property tells the jvm to limit it's cache to 32MiB ( you could also choose a smaller number if it makes sense for your application).

[BrennaEpp]

Closing this due to lack of response. Please open a new issue if you are still experiencing this issue.