[Hermes] : leak in hermes when using rxjs

[freeboub]

Description

Here is a memory leak using rxjs and react native.
A memory leak happens every time ajax().subscribe() is called.
The subscribe function shall be called to reproduce the issue.

This is reproduced only when hermes is enabled.

I also open a ticket in rxjs to forward this information: ReactiveX/rxjs#7193

Here you can find screenshots for hermes usage:

Here you can find screenshots for JSC usage:

React Native Version

0.71.3

Output of npx react-native info

System:
OS: macOS 13.0
CPU: (8) arm64 Apple M2
Memory: 676.73 MB / 24.00 GB
Shell: 5.9 - /opt/homebrew/bin/zsh
Binaries:
Node: 16.14.0 - /opt/homebrew/bin/node
Yarn: 1.22.19 - /opt/homebrew/bin/yarn
npm: 9.4.0 - /opt/homebrew/bin/npm
Watchman: 2023.02.13.00 - /opt/homebrew/bin/watchman
Managers:
CocoaPods: Not Found
SDKs:
iOS SDK:
Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1
Android SDK:
API Levels: 28, 31, 33
Build Tools: 29.0.2, 30.0.3, 31.0.0, 33.0.0, 33.0.1
System Images: android-23 | Android TV ARM EABI v7a, android-33 | Google TV ARM 64 v8a, android-33 | Google APIs ARM 64 v8a
Android NDK: Not Found
IDEs:
Android Studio: 2021.3 AI-213.7172.25.2113.9123335
Xcode: 14.2/14C18 - /usr/bin/xcodebuild
Languages:
Java: 11.0.18 - /opt/homebrew/opt/openjdk@11/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: 18.2.0 => 18.2.0
react-native: 0.71.3 => 0.71.3
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found

Steps to reproduce

Download and install following sample:
https://github.com/freeboub/rxjsSample

yarn install && yarn android

See: https://github.com/freeboub/rxjsSample/blob/main/README.md

Snack, code example, screenshot, or link to a repository

Sample source code: https://github.com/freeboub/rxjsSample

ajax({
      url: 'http://youBigRestApiCall.com/',
      method: 'GET',
    }).subscribe(
      () => {},
    );

[github-actions]

⚠️

Newer Version of React Native is Available!

ℹ️

You are on a supported minor version, but it looks like there's a newer patch available. Please upgrade to the highest patch for your minor or latest and verify if the issue persists (alternatively, create a new project and repro the issue in it). If it does not repro, please let us know so we can close out this issue. This helps us ensure we are looking at issues that still exist in the most recent releases.

[freeboub]

Not sure why I have The bot feedback:

➢ npx react-native upgrade
info No version passed. Fetching latest...
warn Specified version "0.71.3" is already installed in node_modules and it satisfies "0.71.3" semver range. No need to upgrade

[cortinico]

Not sure why I have The bot feedback:

Because you specified 0.70.3 while I believe you're on 0.71.3, right?

[neildhar]

Thanks for reporting this.

Some initial questions:

1. Do you have higher resolution photos of the memory consumption traces showing how much memory is used? Some heap growth is expected initially.
2. Do you have any information attributing allocations? XCode instruments could provide some of this.
3. It looks like Hermes memory consumption levels off at the end of the trace. Does that persist?

[freeboub]

Hello,
Thank you for the quick feeback. and sorry for bad screen shots

1/ Here is another screen shot with higher resolution
The test did 217 GET on youtube.com (in 1O seconds). And you can see that app was launched before the test starts.

3/ no you can see here that memory stays very high after the end of the test:

2/ Here is another screen shot of memory allocation we can see that data is lost in okHttp, but I am not sure it really helps.

Currently I cannot run the sample on ios (not sure why for now, I will have a look this week end or next week).

Thank you

[freeboub]

Addtionnal information, I also have the issue in my production app with react native 0.70.5, hermes enabled / rxjs 6.6.7

[neildhar]

@freeboub It looks like that last screenshot only accounts for 19MB under "remaining size" is that from a shorter trace? If not, I'm curious why that number is so much lower than the native memory stated in the top trace.

If it is from a shorter trace, it would be really interesting to see where the extra native memory is being allocated. Could you expand the primary contributors to remaining size to see where it is coming from? Alternatively, could you share your memory trace here?

[neildhar]

If I had to speculate on the reason you are seeing this, it would be that the library calls you are making allocate some native objects, which are not visible to the GC, and therefore do not trigger collections. JSC likely collects much more frequently than Hermes, and ends up freeing these native objects when it does so.

You can test this hypothesis by adding a call to gc() to force a collection, which should free the excess memory.

[freeboub]

Amazing good solution !
The app was still running with a high memory usage, I reload once memory went high again, then add the gc(); call then the memory stay low !

Anyway I think there is an issue here, but not necessary what I described.
Notice that if I don't stop the loop, memory will go to more than 1GB of RAM and then the application crashes !

[neildhar]

@freeboub Thanks for trying it out and sharing your results.

I don't see an immediate reason to suspect a Hermes bug here, the high memory consumption is likely a result of native allocations being retained by JS which are not visible to Hermes.

However, I suspect this may not be the right way to use this library. Repeatedly performing native allocations in a loop seems wasteful, and there is probably a way to reuse or manually release them.

[DevFuan]

I'm using the subscribe function in an external library, but I think I'm experiencing a similar phenomenon to this issue.
Could you share the logs that occur when the adb logcat --buffer=crash command is executed?

[freeboub]

@neildhar I was on vacation last week, let's continue to describe my finding.
The same issue can be reproduced on ios. Here is the graph in xcode.
There are 2 runs in the graph.
First one with gc() call second one without, and I also see memory increase in second run.

I fully agree this sample is not a real life use case ! but I am trying to isolate components to undersatnd why my app takes too much memory.

I note the gc() trick and keep you updated of my findings.
Thank you

[freeboub]

Hello,
Here are some additionnal findings.

In my first tests with gc(), I calles it after each ajax call -> everything work fine, but too agressive
Then I put the gc() call in a setInterval. Here are 3 different graphs with 1 second, 30 seconds and 1 minute timing.

• 1 second -> looks good, memory stays around 192 MB, but too agressive I think.

• 30 seconds -> We can see that memory is highly growing during 30 sec and then it decrease slowly (nearly 230MB ), but without reaching initial level.

• 1 minute -> We can see that memory is very highly growing during 1 minute and stay stable arround 380MB for the rest of the run.

Not sure my initial description of the issue is still correct regarding to this comment...

@DevFuan I am not able to generate an out of memory crash with the sample, It seems that when I reach 1 GB memory stops growing.

Note: I updated the sample to be able to run gc() with a button in UI

[neildhar]

Merging into #982