Memory usage vs number of leaks across 2 different platforms
emmaxtung opened this issue · 4 comments
I'm running tests of just the initial load of two identical pages across two different web platforms and for platform 1, I see ~80mb of memory usage with 8 leaks. On platform 2, I see ~20mb of memory usage with 11 leaks. Given this information and without looking into the trace (assuming code is minified), would you say that it is possible to make a conclusion whether platform 1 or platform 2 has better memory performance, or would I need to dig deeper?
I was expecting platform 2 to have little to no leaks, so I am surprised by the results, but I'm not sure if low memory usage with high leaks is better than high memory usage with low leaks or not. I'm sure it would be easier to tell if I measured page actions, but platform 2 is not a SPA and therefore I am only testing initial load for both. Thanks in advance
Here are a few things worth mentioning when it comes to evaluating memory performance and memory leaks across platforms and web apps:
- Each test scenario may not necessarily be representative of the overall memory performance of a platform. A test scenario covers a specific subset of operations or tasks, and the memory usage of a platform may vary greatly depending on the tasks it's running. Therefore, having higher memory usage in one test scenario does not necessarily mean that one platform's memory performance is inherently better or worse than the other's.
- Similarly, the number of memory leaks detected also depends on the specific test scenario and may not be representative of the platform's general performance in terms of memory leaks. It's not a clear-cut case of more memory leaks equating to poorer performance. The platform that has more memory leaks in one scenario might have fewer in another.
- The raw number of memory leaks is not always indicative of the impact those leaks have on the system. Different memory leaks can have very different impacts depending on the resources they're holding onto. MemLab clusters memory leaks together and only reports the leak clusters. As a result, it's possible for a platform to have numerous unique memory leak clusters that each leak a small object, while another platform may have fewer clusters, but with each one retaining significantly larger amounts of memory. Therefore, it's not always a straightforward comparison based on the number of leaks alone.
Taking these factors into account, drawing concrete conclusions about memory performance and leaks based solely on the given data can be challenging. It's worth noting that MemLab is primarily designed as a tool for identifying memory leaks and exploring opportunities for memory optimization, rather than as a measurement tool.
A more reliable way to obtain an overview of memory performance is to directly gather data regarding the main thread JavaScript (JS) heap size from the production environment (by using the performance.memory API) and compute the average memory usage as well as the 99th percentile (p99) of memory usage.
In your specific case, you've noted that platform 2 is not a SPA. This means that the JavaScript heap is destroyed and rebuilt after each navigation. While this process may not be ideal for performance due to the constant overhead, it does have a somewhat protective aspect when it comes to memory leaks. Unlike SPAs, where memory leaks can accumulate over time across page navigations, the potential impact of memory leaks in platform 2 might be relatively less severe, as each page load essentially cleans up the JS heap (including memory leaks from previous navigations).
Thanks for the detailed response, really appreciate it! I previously had some assumptions of the factors you listed above so it's good to have this confirmation.