[ListView] renders all rows?
luics opened this issue ยท 82 comments
There're 8 visible rows in screen after refreshing, but there're 15 rows found in chrome react tab.
Will you implement ListView with UITableView in future version?
Thank you.
I wonder if this is related to the default value of scrollRenderAheadDistance:
http://facebook.github.io/react-native/docs/listview.html#scrollrenderaheaddistance
It defaults to 2000 pixels unless you override it and with 15 rows it might not go above that and so will render everything when you scroll? This is just a guess but it might be worth tweaking the props for this component to see if you can affect this behaviour.
React Native is using a different optimization strategy than iOS. Here's a summary
Load balancing
In UITableView, when an element comes on screen, you have to synchronously render it. This means that you've got less than 16ms to do it. If you don't, then you drop one or multiple frames. If you are rendering complex elements like newsfeed stories, it's basically impossible to meet this schedule so you're doomed to drop frames.
With ListView, when you reach the end of the current screen, you can prepare in advance more rows to be rendered. Those rows will be rendered in a different thread so won't freeze the UI thread while processing. The reason why it is working is that the load is not evenly spread. You don't need to render a new story on every single frame, most frames are just scrolling and don't need new stories to appear.
ListView will also render one element at a time, so if you are interacting with some element while rendering more rows, it won't block until all the rows have been pre-rendered, it will only block for one row.
Memory management
UITableView is very conservative memory-wise, it aggressively reuses cells. This decision was made back in the iPhone 1 where memory was extremely scarce. The problem with this is that reusing cell is extremely error prone for the developer. You are given a dirty object, from which you have no idea what mutations happened, and you need to reconfigure it to look like what you want. In our iOS app, this caused SOOO many bugs.
The problem of reusing cell is that some cells have internal state (video player running, text input, horizontal scroll position...) When you reuse them, you need to be able to serialize that state and put it back. This is not always possible nor easy, so you usually either loose this state or it propagates on the new row and causes bugs.
What we found out on React Native is that it is fast enough on iphone 4s to create new cells for every single row. So, we don't need to impose this very hard constraint on ourself. In your screenshot, you noticed that we don't remove rows after you scrolled for a while. That's not entirely correct, we don't remove the virtual dom representation on the React side (what you see in the chrome dev tools), but we do remove those elements from the "dom" and keep their reference.
When they are visible again, we put them back on the dom. In case we have low memory or the list is too big, we may destroy those and recreate them from scratch (loosing the state as mentioned above) in the future. We haven't done this performance optimization yet, but the user code wouldn't be impacted.
We tried to delete the iOS views aggressively but we found out that doing so was actually very expensive. It was better to leave them hanging than to remove them.
Change Detection
In ListView, we have a DataSource object that favors immutability. If you have a list of 1000 elements to render, you want to make those 1000 elements immutable, meaning that you can check the previous one === the next one and instantly know if something changed. This way, when anything change, the only thing you've got to do is to traverse those two lists and do those very fast equality checks and know what rows changed. And then update only those.
Layout
In UITableView, you've got to specify the layout of every single row even when they are not being displayed on screen. So, in cases where it's not a fixed size, you've got to basically render the element to know its size, and pay that high cost up front. It's also very annoying to do so manually.
In ListView, since React Native owns the layout system, you don't need to do all that painstaking manual computation yourself. When a row is rendered, it'll update the size. The only downside is that the scrollbar is a little funky, but I'm sure we'll be able to come up with heuristics to smooth it out in the future.
Something doesn't seem to be working as intended just yet. After scrolling a very long list view with images, and 4 rows visible in vertical ipad air 2, everything seems to be working as @vjeux describes, except the memory just keeps climbing and climbing. And the most alarming thing is that scrolling (after scrolling to the bottom), can cause main thread to easily eat over 100-130% of cpu, while the JSCore thread stays quiet. This is very different to the 1-3% taken by just JSCore when scrolling from the top after the initial render. It is very apparent that the app is unstable while the listview's rows all remain in memory, and the main thread is blocked for about a second give or take while a listview in this state is unmounted, and can even result in a crash. Just a reminder, this is all on an iPad Air 2.
I also tried the "experimental feature" of removeClippedSubviews for ListView. This initially did seem to help with memory, but it was causing a crash. I assumed this was the "experimental" part, and haven't dug in there just yet ;)
What I'm wondering is if I am just missing something? Do I have the capability to remove images myself using onChangeVisibleRows, should I consider this? Would it be worth more getting removeClippedSubviews stable, or should I start contemplating a new version of list view with recycling?
Does Instruments provide any insight into what's leaking? (specifically wondering if the problem is in JS or Obj-C)
@ide I'm a noob at instruments profiler... So here's brief summary from me taking a look at it.
cpu profile looks like most of the time is spent here (recursing through subviews), in RCTView.m :
- (void)react_updateClippedSubviewsWithClipRect:(CGRect)clipRect relativeToView:(UIView *)clipViewIn memory profile major causes of persisted memory (645 MB total) are:
- VM: CG Graphics Data (410 MB)
- VM: CoreAnimation (141 MB)
- VM: JS Garbage Collector (61 MB)
- ...
Unmount the ListView component, and total persisted memory drops to 74 MB total:
- VM: JS Garbage Collector 58 MB
- VM: CoreUI image data
- ...
I have to think I'm doing something wrong, or maybe the ListView just isn't prepared for what I'm throwing at it.
Those profiles were from attaching to iPad Air simulator.
cc @bryceredd who's been investigating React Native performance
Closing this since it's not very actionable.
@vjeux What would make this actionable? I asked a few questions in this thread that was answered by closing the issue? I've also still been digging through the innards of the ListView implementation, as well as the crash caused by removeClippedSubviews: true. The title "ListView renders all rows?" seems appropriate based on memory consumption no?
Sorry, trying to clean up the hundred plus issues we had, I was a bit too quick on this one.
@vjeux No worries, and thanks ;)
What I've found so far regarding the removeClippedSubviews: true crash
I'm seeing it happen on line 217 of RCTScrollView:
UIView *nextHeader = nextDockedIndex >= 0 ? contentView.subviews[nextDockedIndex] : nil;When I change the data source to contain only one section (same amount of rows), this crash does not occur, and I can see that the views are being clipped.
In this case, the memory is lowered to about 1/2-2/3 what is was before, but still very high, and cpu is not spiking as it did on minimal scroll, also about 1/2-2/3 of what is was before.
It's totally insane: my iPhone 5c crashes after 700 list items. If I'm going to write a chat - it's blocking for me.
Also I got a lot of "Cannot find single active touch"
removeClippedSubviews is our current stopgap that makes the groups app work - can you guys try scrolling in that app and see what you think of the perf/stability?
We definitely need to fix crashes when you have removeClippedSubviews - I don't think we tested thoroughly with sticky headers (they aren't used in groups), so that might be the issue.
Note on OP - the react tab just shows the react elements in JS. Just because they show up there doesn't mean they are in the actual UIView hierarchy (and you should find they are not all in the UIView hierarchy if you enable removeClippedSubviews).
my < ListView pagingEnabled={true} onEndReached={this.loadAnotherFiftyArticles} >
50 rows/pages of < Image / > < Title/ > < Description/ >
3rd Load append (total 150)
Received memory warning
Received memory warning
Received memory warning
Crash Physical iPhone 6 Plus
Also I got a lot of "Cannot find single active touch" time to time
๐
Hi all
i create the PR #1406
It's solve for me the issue if you have a better idea to solve it feel free
Thanks
Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.
I took a stab this.
In my perf testing I was seeing constant memory allocations using ListView as the user scrolls down long lists. This is obviously problematic for reasons that don't need to be explained.
In response, I created an NPM package react-native-sglistview: https://github.com/sghiassy/react-native-sglistview.
The package isn't where I want it yet - but its showing some process
@sghiassy This looks interesting, will def check it out.
Can't find a way to make the ListView component work well pulling data from Firebase javascript client.
Here's what I'm trying to create:
- Add data when the scroll reaches the bottom (infinite scroll functionality should be pretty easy to do)
Here's how I'm doing it:
- On componentDidMount call function that fetches the data. (./app/components/tabviews/home/homefeed.js)
- I use blackbird to return a promise with data returned from Firebase.
- When data gets returned setState for dataSource using cloneWithRows(data).
So far I was able to display some of the rows, in the first call the promise returns 50 items but only 25 rows are displayed. I can't reach the bottom! If someone wants to help me it would be great because I've pulling my hair out to make this work. Thank you!
Here's my code:
I can vouch that @sghiassy's https://github.com/sghiassy/react-native-sglistview is a great solution at the moment. I've been using pretty much the same logic (in production) but never found time to pull it out into a separate library. Great work by Shaheen.
It completely destroys the views when they are out of screen and hence removed from memory. removeClippedSubviews will only help scrolling smoothness (GPU optimisation) - it is not a memory optimisation. Objects/views/images are still in memory and hence the memory will still climb.
@alejomendoza Try disabling onEndReached. You are basically fetching the same data again on reaching the end, so it overwrites all the row's it has rendered till now. So effectively when it reaches the end, it starts over.
You can add a console.log in renderRow and render to debug that it indeed starts rendering but then starts over on reaching near the end. The third parameter to renderRow is the rowID, which you expect to be able to climb upto 50.
thanks @paramaggarwal ! I debugged it and I get all the items but only 9 are rendered, saw this through adding console.log in renderRow, when I scroll to the bottom it renders the 50 items but it wouldn't let me scroll further down. Here is the video: https://vid.me/tJum https://github.com/skyhitz/skyhitz-react-native-ios
@alejomendoza I have seen your issue before it has to do with the container views holding your views not having flex: 1 I'm not home right now but later I can provide you with an example so you see what I mean. In the mean time try adding flex: 1 to all views wrapping your listview
thanks @josebalius ! I tried adding flex:1 to all container views and it worked!
Here's a version that unmounts rows when they are out of the screen and more context from @sahrens
This type of "buffered" pattern can only mean very good things for the community! Absolutely awesome job, @sahrens!!
Thank you so much @vjeux. That's a really nice diff and great comments from @sahrens.
While reading I realised, that we as a community think about a solution by building something on top of ListView itself. I never considered modifying the ListView implementation itself like you do above. Nice!
@sahrens The removeClippedSubviews implementation you talk about is different from the one in the OSS RN repo right? The one in the OSS repo simply removes from view hierarchy and still will continue to eat up memory. The optimisation by @nicklockwood is where it goes further and removed images and text from the memory too, right? (ref: #332) Looking forward to it in the OSS repo. Will be of much needed help.
Thanks you guys!
@paramaggarwal the removeClippedSubviews feature is the same internally and in OSS, but the implementations are different. @a2 recently added the feature of unloading offscreen images and rendered text to OSS, which should hopefully help with memory.
๐ Would be nice to see those changes in OSS RN :)
@michaelcontento they are included in v0.10
@nicklockwood Will check it out.
@a2 that's a really smart solution to the problem!
Thank you so much! :)
All, here are the relevant commits:
I guess we can close this issue now.
Our app has two tabs of quite long list view, switching between tabs is getting much slower compared to previous versions. We haven't enable the "removeClippedSubviews+overflow:hidden" optimizations yet though.
Having problems with an iTouch 5th gen and ListView gradually filling up with memory. Over 1000 rows, followed this thread but nothing is making a huge difference at the moment. Tried working with a native table view implementation and the app took ages to load the data. Seems more sensible to group the data in chunks and drill down at this stage - although I know that is basically just a workaround. When I've got more time - I'll have a look at this as well.
How would android-platform like?
Hi @paramaggarwal @sahrens ,
As mentioned in above comments , I have implemented Listview , My application is kind of Offline Dictionary which has .sqllite DB with lage number of words.
I render 50 words each time with the offset on every scroll ends , even after setting up removeClippedSubviews to true , my RN application is eating up so much memory.
The below shows that it has not more than ~500 records in listview it is eating up huge memory.
I also tried @sghiassy but it seems doesn't solve my problem.
Could you please provide some some help ?
tried all tricks mentioned, 300 rows crashes every time.
Does each of your rows have overflow: "hidden" which is needed to work along with removeClippedSubviews.
@paramaggarwal thanks and yes. It's added to each row. shouldComponentUpdate implemented. Using rowHasChanged as the doc suggested. Infinite scroll implemented.
Memory usage boost up to 500MB and crashes at about 800 rows.
PS. Each row contains a feed item with some text and images, similar to the fb groups app.
So removeClippedSubviews will help only if your rows contain images and text. If there is any other kind of view, it will not be automatically removed from memory. (All views in the row would be removed from the view hierarchy though.)
@paramaggarwal thank you. My row item contains only text and images. As you mentioned before, 'removeClippedSubviews will only help scrolling smoothness (GPU optimisation) - it is not a memory optimisation', I assume that the ListView will eventually crashes the APP as number of grows, with or without removeClippedSubviews. Is that right?
Since RN 0.10, removeClippedSubviews will help with memory also as images and text are actually remove from memory when they are taken out of the view hierarchy - which is what it does. Hence this particular memory issue for large number of rows is actually fixed since then for the majority of use cases.
@paramaggarwal thank you. I will do some more tests to see what's wrong with my code. Good day.
Hi @paramaggarwal , I am coming back with some test results with RN 0.14.2 :)
On my iPhone 6, the app will crash after the memory usage is over 500mb. The following code will crash the app if the number of rows grows to 1800 with removeClippedSubviews off, and 5200 with removeClippedSubviews on.
In another test with some images in the row, it crashes at 300 rows with removeClippedSubviews of, and 600 rows with removeClippedSubviews on.
Can I assume that this is the best ListView can do at this moment?
let React = require("react-native")
let {
View,
Text,
ListView
} = React
let itemsPerPage = 20
let ListViewTester = React.createClass({
getInitialState: function () {
this.data = []
return {
dataSource: new ListView.DataSource({
rowHasChanged: (row1, row2) => row1 !== row2
})
};
},
componentDidMount: function () {
for (let i = 0; i < itemsPerPage; i++) {
this.data.push({
index: this.data.length
})
}
this.setState({
dataSource: this.state.dataSource.cloneWithRows(this.data)
})
},
_onEndReached: function () {
for (let i = 0; i < itemsPerPage; i++) {
this.data.push({
index: this.data.length
})
}
this.setState({
dataSource: this.state.dataSource.cloneWithRows(this.data)
})
},
_renderRow: function (row) {
return (
<View style={{height:44,overflow:'hidden'}}><Text>
Item {row.index}
</Text></View>
)
},
render: function () {
return (
<View style={{flex:1}}>
<ListView
removeClippedSubviews={true}
dataSource={this.state.dataSource}
onEndReachedThreshold={0}
onEndReached={this._onEndReached}
pageSize={10}
renderRow={this._renderRow}/>
</View>
)
}
})
module.exports = ListViewTester
Hi,
I agree with @kinhunt: the current implementation is unable to deal with big amount of data, memory use increases and then crashes the app...
I've made the same test with NativeScript's ListView (using the same data) and memory was always around 100Mb and the app didn't crash...
Stefano
Hello
Just an observation...
When running @kinhunt 's sample I am noticing a large difference between the way memory is managed on actual hardware vs the simulator. I guess this is to be expected because the simulation is just that, a simulation, however this seems to be really drastic.
In this test case I am using @kinhunt 's example code, and am scrolling approximately 1000 rows each for each test. Memory values are from xcode's debug navigator panel.
- Hardware Test Results - iphone 6
Memory usage goes from 54mb at the start of the scroll and climbs up to 163mb before leveling off at 135mb. - Simulation Test Results - iphone 6
Memory usage begins at around 54mb and only climbs to 66mb after scrolling 1000 rows.
Should this much of a difference be expected? I can also confirm this same behavior on another app I am developing.
Nick
Please report on https://productpains.com/product/react-native/
I've been working on an experimental WindowedListView that should help with
this problem. I'll try to get it out soon.
On Tuesday, December 1, 2015, Param Aggarwal notifications@github.com
wrote:
Please report on https://productpains.com/product/react-native/
โ
Reply to this email directly or view it on GitHub
#499 (comment)
.
@sahrens That's awesome, looking forward to it.
@sahrens great! expecting.
@sahrens +1
@brentvatne: how's your fixed row height fork of WindowedListView coming? Are you going to contribute that back? I think that's probably the best option for most cases.
I'm still stalled on getting windowed list view with variable height rows working with Incremental.
@sahrens @brentvatne Can we see the work-in-progress, and possibly help out? ๐
Also, could any relevancy be gleaned from bvaughn/react-virtualized? I know it's not yet setup for React Native, but a lot of the modules seem useful (eg: VirtualScroll).
Thanks for all your work, guys! ๐
This is a serious issue. It makes the app unworkable :( +1
+1
Would love to see the update come out soon! Thank you for working on it Im real excited!
My colleague is currently working on re-writing our RN ListView into a fully native view (we probably wouldn't have used RN at all had we initially known about this performance problem). In the tests I've seen him done, it's MUCH smoother. No promises, but I'll see if we can open source it.
@sghiassy's https://github.com/sghiassy/react-native-sglistview is still a good alternative. I'm using that currently in my app. I've reduced my background image to 1/3 of the size, this allows smoother frame rate and much less usage of the memory. However, I still prefer the native solution if possible.
@jamesfzhang funny you should say that, and an FYI to the RN team. This issue, this component, and the initial comments on this thread, were a huge reason why we did not go with RN. Yes most of the framework was pretty awesome, but this particular component is a huge misfire, and it was disappointing to see the debate that seemed to favor its implementation.
I'm afraid we didn't do a great job of communicating our position on this.
We created React Native to solve our own app development requirements (which include long scrolling lists of content) and when we found that ListView was satisfactory to meet our requirements, we prioritised other features (startup time, network and image loading/decoding performance, etc) over improving it further.
But that is not to say that we don't recognize its deficiencies, and now that we have met our performance and functionality targets in other areas of the framework, improving (or replacing) ListView has moved up to the top of our priority list.
If you are struggling with sluggish performance or out-of-memory issues when using the current ListView for hundreds or thousands of rows, you can expect to see some improvements from us in the near future.
That said, if you are already working on your own solutions, I encourage you to continue doing so, as there are many different types of requirement for this type of view, and we may not be working to solve exactly the same problems that you are.
@nicklockwood Thanks. I appreciate the thorough, timely, and very open response to my comment. Just to make sure you and the team knows it, whenever I make a comment like that, it's always meant as constructive criticism, I know you took it like that, so thank you. Just because we decided against RN for our needs, does not mean it isn't impressive piece of open source software. So keep up the good work, and I'll look forward to seeing the direction this component goes!
@drkibitz understood, and thank you for the constructive feedback - we do take this stuff on board, even if we aren't always in a position to act on it right away.
@nicklockwood Thanks for offering more info on that, much appreciated. The truth is that we absolutely love RN and will look forward to seeing the performance improvements. Thanks and keep up the good work!
Hi, i wondered if the issue is the same in the android part? And android part is using recyclerview right? And it is not using the recycle part?
And I think maybe for best is theres 2 implementation of listview. The react way and the native recycle way for lot of items
Just my 2cents
In belated response to @aleclarson's comment, I've been moving more and more of the core windowing logic for react-virtualized into util functions. I've wondered a little if it would be useful to abstract things further and maybe even move it out into its own package but I didn't know of any immediate uses and I didn't want to spend time doing it just for the sake of doing it.
That being said, if there's any interest or chance to contribute here I'd be happy to.
Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/listview-renders-all-rows
ProductPains helps the community prioritize the most important issues thanks to its voting feature.
It is easy to use - just login with GitHub. GitHub issues have voting too, nevertheless
Product Pains has been very useful in highlighting the top bugs and feature requests:
https://productpains.com/product/react-native?tab=top
Also, if this issue is a bug, please consider sending a pull request with a fix.
We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.
I worte Real Android recycles RecyclerView/ListView,
To solve the problem of RecyclerViewBackedScrollView.
any news from react native team about that?
Is this issue will be fixed in any release?
"removeClippedSubviews = {true}" saved my day!!
removeClippedSubviews work on ios, but don't work on android
@droidwolf according to this 25cd2c5 it should def be working on Android, where do you draw that conclusion from?
ListView experience was not very goodใ
see the RecyclerViewBackedScrollView source file.
the mViews variable hold all item views in memory.
WindowedListView is working fine
windowedlistview does not well on Android see here: #11950 (comment)