Virtual scrolling helpers / utilities.
jelbourn opened this issue · 94 comments
Similar to the md-virtual-repeat in Angular Material 1.
Would be good to implement dynamic height (angular/material#4314)
Does someone know a repo do this virtual scroll instead of ngFor ? Today we have a very large set of data, it's primordial to begin/HAVE this feature instead of something like "tooltip" or so ! @jelbourn Displaying big data is important today please begin with something hard instead of "tooltip"
ps: https://developers.google.com/web/updates/2016/07/infinite-scroller
@istiti Polymer has <iron-list> Here and Demos Here
Please recognize that you are asking about a feature that most projects don't need/use and is a technical challenge to put into a library in a fully formed, universal way... You could probably write your own version, but it might not be what I would need...
Tooltip is a super easy use case for portals and overlay's which are core to everything from menu's to dialogs so is rather simple to release and common enough that most use cases fit.
The link you gave has a link to his source code and it's only 400ish lines. I bet you could make a ng2 version and contributing it back to the community for free would be awesome!
@DennisSmolek "Infinite scrollers pop up all over the internet. Google Music’s artist list is one, Facebook’s timeline is one and Twitter’s live feed is one as well"
IMHO infinite scroll is technik people need to have to discharge their DOM! Still more today we have big data.
Update
Impossible to get iron list working (very comlpicate) if you have an github repo doing virtual Dom, you're welcome
yes but this add more complexities to my app ... really a bad choice but I find angular2-data-table which do work very awesome thanks
ETA?
I've ported the old one and wonder if I there's some form of spec available so I can align my implementation with it. The mdVirtualRepeat is almost (as in line-to-line) the same, but structural repeater had to be made by scratch based on ngFor and mdVirtualRepeat.
@Klaster1 , any chance you could fork material2 and add your work to the library? If there was at least an initial PR we could get some momentum going behind this.
ngx-datatable has an implementation of virtual scroll.
I wonder if that couldn't be easily brought in?
@jelbourn
I've made a naive implementation of a templatizable-virtual-repeat that work great for basic use-cases.
However, the item repeated must have a fixed height.
For now the virtual-repeat only supports:
itemNumber * itemHeight / maxHeight of HTMLElement of the browser which is 33554400px in Chrome.
So the number of items to be virtualized is still limited and relative to their heights.
If this sounds good to you, I can make a PR with the initial version and if the components pleases most I can work on the maxHeight issue.
Let me know if you're interested.
@ThibaultSavary we want to explore solutions for non fix-sized items in material2.
@jelbourn is that the sort of thing that can be implemented in a version bump down the road? Variable height is a tricky issue to solve with virtual repeaters, and I'm not sure insisting upon its inclusion is a realistic goal.
If we could at least get feature parity with Angular Material 1.0, it would be a massive help; Our company is pretty dependent on md-virtual-repeat, and its lack of support in Angular Material 2.0 is a bit of a blocker for us to upgrade to Angular ^2.
To be sure, flexible height would be a great feature, I'm just not sure it's as much of a requirement as having virtual repeat in the first place.
@jelbourn I fully agree with @bryanrideshark for virtual scroll to be exact in height is not top priority. Exact position affects scrollbar position on the right, but normally it is only a visual effect.
@bryanrideshark & @laserus I don't agree with you guys... if the team implements the v1 (from material 1), a better version will probably never be explored! The team's time is limited and they should focus on innovation rather than existing technologies
+1 for innovation
ps:wait this feature since 5 months
+1 for the flexible height.
@natcohen all of what will / will not happen is speculation unless we're actually decision makers in the matter, IE, core members of the team.
As this is a blocking issue for other things (autocomplete, md-select), would someone on the team be able to provide some sort of feedback as to where this issue stands?
+1 for the flexible height
+1 need this badly!
I have an implementation for virtual grids: https://github.com/Hacklone/angular2-cool-infinite-grid
Maybe grids could be part of virtual scrolling too :)
there aren't virtual scroll component In milestones. Are you planning to implement it?
https://github.com/dinony/od-virtualscroll is working great in my project, and so far is the only one I've tested that I could get working (the others might not have liked @angular/flex-layout, not sure).
It doesn't yet support dynamic item height, so for now we're just using it in a feature branch, but otherwise seems like a really good implementation.
@mackelito, yep, they seem to be working well together; I just had to include a small CSS workaround. This is what my implementation looks like:
od-virtualscroll, od-virtualrow {
width: 100%;
}
...
<div fxFlexFill fxLayout='column'>
<od-virtualscroll fxFlexFill>
<ng-template let-item let-row='row'>
...
</ng-template>
</od-virtualscroll>
</div>
Also potentially helpful (not material/flex-layout-specific): dinony/od-virtualscroll#2 (comment)
+1 for the flexible height.
What do you mean by flexible height,
Do you expect the repeater to calculate the height by itself and assume that all items has the same height
or you wish for every item to have it's own different height?
I am asking because I already implemented this component and the former wish is crazy.
The first is simple though, before creating the components loop, just build the first component and then calculate it's height...
@MadUser customizable height calculation function might be good enough. Default implementation will measure first item or use number value from binding/config object. If you can calculate each item dimensions reliably, variable size will work. For example, if each virtual list item is another list of items of predefined size, just multiply item height and items amount (in simpliest case).
I'm currently using this one: https://github.com/rintoj/angular2-virtual-scroll
Works great
Any updates on this?
@elineopsommer they have updated the feature status in the info page to "planned Q3 2017"... Q3 has just started so just be patient :)
@elineopsommer where is the info page?
Huh after reading through the comments i am surprised how complicated you guys want to make it. Isn't enough just to set the scroll distance for triggering an event to load more data. I see no point of being unable to be innovative with this. A working version that works simply should be enough and covers the most use cases. You can then be innovative and add more features?
Some people disagree, @azarus. Why would you be on a team that is commited to create something with the best design principles and will just ignore this fact to create something that is already there? If your use case is simple, just work with some third-party library (or make a component) that fulfill your needs. The team cannot just start with something simple now, before designing with all cenarios in mind, as this may lead to a bad/wrong spec to other more complex cenarios.
Following your example, there is already other projects that can do what your want. For instance: https://github.com/rintoj/angular2-virtual-scroll. This is a nice implementation, but doesn't follow Angular Material principles. I think that's why the team is taking "so long" to implement the entire library with the best UI components in mind: design everything really well.
In a project that I am participating now, an use case for the virtual repeat component is a grid of images, that I load records from a service. My intent with this is to show it to the user like I would do using a RecyclerView (the Android component). And, in my opinion, this is where virtual scroll could be taken: a "mimic" of a RecyclerView that allows us to display an interesting UI without loading all the records upfront, but with the items "size" already defined (as we can run a simple count in database to check this quickly). This will help us to deliver better UI/UX to the user: a nice component to interact with and with great performance too.
The scrolling is already implemented in material but there are no such events yet as far as i see :/
I think we aren't talking about the same thing here, @azarus. This issue is about tracking the Virtual Scroll/Repeat implementation for Material UI of Angular 2. It's not implemented yet. Not even designed.
The only news from it is an update (a couple of days ago) regarding the plan to start designing this component, that was postponed to Q4 this year.
Anyway... I have a lot of expectations about this component and hope to see it covering complex use cases too, not only what we already can do with any UI library today. This is what I think the team behind Material wants too and the work to polish everything is something big, but the results tend to be very promising.
One year later...
Just wait for them as they said they gona reinvent the wheel in an innovative way. :) such a joke.
@azarus as one that is inspired by the Material team and their work I take comments like this as hurtful to this community. I understand you're frustrated with the lack of progress on this particular issue, but the team has been working their ass off on bringing core functionality with i18n, a11y, and the cdk. Your comments have added zero benefit to this thread.
I ask that you please resign from commenting.
I found brilliant solution from slickgrid. This is basically handles unlimited records of any size (I don't think variable height can be handled in this case) and I have implemented the solution preview, comments added. This solution performance is too good compare to any solutions exist today. May be material team can consider to adopt it?
@cgatian then stop mentioning me in your fan comments? I do like material but this is slightly anoing how no progress is shown regarding this feature. If a complaint is hurtful because the team puts resources into reinvention rather than delivering it then it better be damn good?
@azarus +1
It's like google is very cool and have no profit to make all developer use angular or other. All is open source open source open source open source lol
This thread need tobe blocked
Yep, need to be blocked until it will be done or in progress.
Also take a look at https://github.com/tbranyen/hyperlist , non angular pure js version.
@jelbourn I know this isn't planned for post beta but was wondering if there was any API proposals floating around for this?
I've been looking at a few different ways this could be implemented but wanted to check to see if what I've come up matches anything that the team might already have in mind.
Hello,
Here is my own implementation of the virtual-scroll, inspired from one of AngularJs but coded by me, it does not support dynamic height of items but has really great performances (works perfectly well on CPU 6x slowdown with a fade-in animation for items).
Usage:
<optimized-list [items]="products" [itemHeight]="73" [offset]="2" (update)="optimizedList = $event">
<!-- It supports animations: -->
<div [@reveal] *ngFor="let product of optimizedList">
{{product.name}}
</div>
</optimized-list>
nb: "offset" represents the number of element preloaded (item rendered but not in the view) to prevent "pop-in" effect. This parameter is optional and default to 0, it's completely OK to keep it at default but you should add a fade-in animation. A really good compromise is to set it to 2 with a fade-in animation. It performs really well on low-end devices.
component.html
A "fake-scroller" is an empty box in which the user can scroll, it has the total height of the list, the content is not in this "fake-scroller" but on top and will be translated using the css translateY function.
<div id="fake-scroller" [style.height]="scrollHeight + 'px'"></div>
<div id="content-wrapper" [style.transform]="'translateY(' + translateY + 'px)'">
<ng-content></ng-content>
</div>
component.ts
import { Component, OnInit, OnChanges, SimpleChanges, Input, Output, EventEmitter, Renderer, ElementRef, HostListener } from '@angular/core';
@Component({
selector: 'optimized-list',
templateUrl: './optimized-list.component.html',
styleUrls: ['./optimized-list.component.css']
})
export class OptimizedListComponent implements OnChanges {
/*
My own implementation of a virtual-scroll component
~Robin
*/
@Input() items: any[]; //the items to display
@Input() itemHeight: number = 20;
@Input() offset: number = 0;
@Output() update: EventEmitter<any[]> = new EventEmitter<any[]>();
@HostListener('scroll') onScroll = function () {
this.refresh();
}
public scrollHeight: number;
public translateY: number;
private visibleItems: any[] = [];
constructor(
private renderer: Renderer,
private elementRef: ElementRef,
) { }
ngOnChanges(changes: SimpleChanges) {
for (let propName in changes) {
if (propName == 'items') {
this.scrollHeight = this.items.length * this.itemHeight + 88; // + 88 // offset bottom
//This offset is not necessary but I think the list should support both top and bottom offsets via parameters
this.refresh();
}
}
}
private refresh() {
let height: number = this.elementRef.nativeElement.clientHeight;
let scrollTop: number = this.elementRef.nativeElement.scrollTop;
let visual_start: number = Math.floor(scrollTop / this.itemHeight);
let offset_start: number = Math.max(Math.floor(scrollTop / this.itemHeight) - this.offset, 0);
let offset_end: number = Math.min(Math.ceil((height + scrollTop) / this.itemHeight) + this.offset, this.items.length);
this.translateY = Math.min(scrollTop - (visual_start - offset_start) * this.itemHeight - (scrollTop % this.itemHeight), this.items.length * this.itemHeight);
this.visibleItems = this.items.slice(offset_start, offset_end);
this.update.next(this.visibleItems);
}
}
component.css
Nothing special about the css:
:host{
display: block;
position: relative;
overflow: hidden;
overflow-y: scroll;
will-change: transform;
}
#fake-scroller{
height: 100%;
}
#content-wrapper{
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
}
@mmalerba is working on it- it's a large project, so it was started in Q4 and is planned to wrap up in 2018.
@jelbourn will it support dynamic height of elements? It is a really important feature in my opinion.
Yes, that is something we aim to support (and one of the reasons it's a larger undertaking)
My 2 cents, virtual list should handle the following:
- Dynamic height.
- No limit on number of elements in the list:
Every browser have max pixel limit, so if list element height say 50px the max elements can be rendered would be approximately 214749 elements (considering the lowest max pixels). Read the article here
const div = document.createElement('div');
const style = div.style;
style.position = 'absolute';
style.left = '99999999999999px';
style.top = '9999999999999999px';
document.body.appendChild(div);
const size = div.getBoundingClientRect()[left ? 'left' : 'top'];
document.body.removeChild(div);
return size;
}
/* Example outputs
Firefox: 33554400px
Chrome: 33554428px
Opera: 33554428px
IE Edge:10737418.23px / IE 9: 21474836.47px
*/
- Support both vertical and horizontal scrolling.
- Should allow filtering.
- Dynamic list size (number of elements).
- Scroll to element if user input the element index.
I potentially see the two solutions angular material team can consider or refer to the one mentioned in the above article (this solution can't handle dynamic height) and another here which can handle dynamic height too).
I've really enjoyed working with the angular2-virtual-repeat interface. It's easy to understand and implementation is a breeze.
A request I have for the interface would be an easy way to shift the elements in the viewport by an integer to support arrow key and page up/down navigation. I have a use case where users can select an item in the list and can use the aforementioned keys to change the selection. A challenge is keeping the selected item always visible in the viewport. This would ideally operate similar to how the material2 autocomplete component works for arrow key navigation.
I've implemented this with angular2-virtual-repeat's scrollInto function, so an option like that would also work.
If it helps here are two different implementations of Virtual scrollers I made a little while ago just to see how the different methods might be implemented.
Intersection Observer
https://stackblitz.com/edit/angular-virtual-scroll-passive
Scroll watcher + manual height calcs
https://stackblitz.com/edit/angular-virtual-scroll
Reload the preview window when you open them up
Just note these are very buggy and have issues, I made these to experiment not as a final product.
Ionic and Iron-List both have Grouping / Divider.
Grid support would be also great (recalculation on resize - means absolute position).
The most advanced virtual scroll implementation I saw up to date is Google Photos
- Grid support - images
- Variable width !
- Grouping
The best available one is iron-list.
@jelbourn Any updates on this?
I implemented an infinite scroll behavior, where every time the scroll comes to the end of the list I make a request for my server and load more information and give an append in my list, however when passing 400 records my list gets very slow .
Any idea what I can do?
the implementation was done natively, without using external libraries!
Sorry for my english, it's not my first language.
@Jonas-Ceolin-Alberton you can check out the projects page > virtual scrolling project in this repo for the WIP on this issue. It's being actively worked on.
thank you @thefliik !
@Component({
selector: 'my-table',
templateUrl: './my-table.component.html',
styleUrls: ['./my-table.component.css'],
providers: [],
changeDetection: ChangeDetectionStrategy.OnPush
})
For anyone stumbling upon this thread via Google because your mat-table freezes, you can try updating changeDetection to OnPush. In my case, I have a lot of *ngIfs in my rows, and profiling in Chrome shows that Angular is trying to check for changes for a minute or so (I have about 900 rows). Using OnPush fixes this problem.
Of course, it's not a silver bullet to all mat-table performance issues, but it might be worth a try in your project. With this option enabled, you might be required to inject ChangeDetectorRef and call markForCheck()/detectChanges() yourself. See @PascalPrecht's helpful article for more details: https://blog.thoughtram.io/angular/2016/02/22/angular-2-change-detection-explained.html
Here is a demo of the virtual scroll with autosize for those interested - https://stackblitz.com/edit/autosize-gap-scroller-ftwx3x
@amcdnl Thanks for sharing this, I think all the elements are already added in DOM. I believe in virtual scroll the elements should be added dynamically depending on available space.
I think all the elements are already added in DOM
No they arent...Did you look at the DOM? Only visible + buffer.
Yes, Please take a look:
The first time I loaded the stackblitz I saw the behavior reported by naveed, but later on when I reloaded, I no longer saw the issue. I didn't investigate further but presumed either there are some outstanding bugs here, or that there was some weird caching issue on stackblitz's end.
I can second @naveedahmed1 experience. Using chrome's inspector, I count 50+ divs matching .cdk-virtual-scroll-content-wrapper > div in the dom---which appears to be all of the possible virtual scroll elements. Though maybe I'm misunderstanding something. (using chrome 66).
(also, each of the divs contain content in them)
Weird; when I first load it it has all the elements as reported by @naveedahmed1 . But if I change the stackblitz to have a loop to 1000 instead of 10 then it behaves the way @amcdnI says it should. And if I then click Fork and reload the stackblitz, it gets lots of errors in the console (altho it doesn't have the extra elements anymore).
@takamori ah, ok. I'm able to replicate that. If I increase the loop to 1000 it starts behaving as expected.
weird problems when i increase
In the final release it will be possible to specify the total size of the data set, like in the AngularJs virtual repeater (getLength())? This way the scroll bar can be initially sized to allow an easy access to any part of the data set.
+1, can't wait til this is ready :)
@byronsanchez - You can use it today in cdk-experimental
For the sake of us joining this years-old item late, can anyone point to any currently available code (presumably a combination of CDK experimental and outside code) which simultaneously provides the following?
- Uses
mat-tableand friends - ... to comply closely with Material appearance
- virtual scrolling, that is, only have the detailed cell-by-cell DOM exists for visible rows
- infinite scroll, that is, can handle a row set of unknown length, where additional data is provided from outside (typically a server, but it wouldn't matter from the point of view of the grid implementation) as needed
?
@kylecordes - The next phase is to integrate this with the other components such as table and tree. Stay tuned for updates on that.
@kylecordes Take a look here https://virtual-grid-mat.stackblitz.io/
Anyone tried it with Angular Universal?
Can anyone please confirm if the following features are supported in current implementation:
-
- scrollTo(position) - scroll to the given scroll position
-
- scrollToIndex(index) - scroll to the given index in the list
Secondly, is it necessary to set height of the cdk-virtual-scroll-viewport? In my case if I dont set the height it doesnt work properly.
No idea for scrollTo/scrollToIndex.
I can answer for height. I set my virtual scroll viewport using flex layout and it work with no fixed size.
See this https://stackblitz.com/edit/angular-virtual-scroll-and-flex-layout-working
Thanks for sharing this @bsthilaire it seems that this solution doesn't work with autosize, I mean for dynamic heights.
I modified my stackblitz for autosize and it's still working with flex-layout.
https://stackblitz.com/edit/angular-virtual-scroll-and-flex-layout-randomsize
I am doing tests on IE11 using stackblitz of @bsthilaire for now.
first bug:
Blank zone at the end of the scrollbar (sometimes) not present on CHrome and Edge
IE11 is glitchy. Do you prefer to have a dedicated Issue to track all problem about IE11? or do we go one by one with different issues?
--> i am going to try to fix this one next weekend.
@amcdnl it seems that the scrolling through arrow keys (up/down) doesn't work properly. It does scroll for few items and after that stops working.
To replicate the issue, open https://stackblitz.com/edit/angular-virtual-scroll-and-flex-layout-randomsize click on any element of the list and then use arrow keys to scroll.
cc @mmalerba
@nasreddineskandrani The gap at the end is alread tracked by #11195
@bsthilaire thanks! i referenced this issue in the issue you mentionned. Since like that we can see the status here :).
If there is other issues opened not referenced here and you know about => please tag this issue the other side to be able to have a clear view on how mature this feature is. And also maybe help :)
I've been tracking all of the virtual scroll issues in this project: https://github.com/angular/material2/projects/20
Sorry for the lack of responsiveness, I just got back from a long vacation, will resume working on this now that I'm back
Anyone tried infinite scroll with cdk-virtual-scroll? I wanted to know what would be the best approach, I tried this:
<cdk-virtual-scroll-viewport #virtualViewport class="demo-viewport" autosize
(scroll)="onScroll($event)">
And
onScroll(e) {
const viewHeight = e.target.offsetHeight // viewport: ~500px
const scrollHeight = e.target.scrollHeight // length of all table
const scrollLocation = e.target.scrollTop; // how far user scrolled
// If the user has scrolled within 200px of the bottom, add more data
const buffer = 200;
const limit = scrollHeight - viewHeight - buffer;
if (scrollLocation > limit) {
this.loadList();
}
}
This seems to work, but has performance issues, so in order to optimize the performance I tried adding debounceTime.
@ViewChild('virtualViewport') public virtualViewport: ElementRef;
ngOnInit() {
fromEvent(this.virtualViewport.nativeElement, 'scroll')
.pipe(
distinctUntilChanged(),
debounceTime(250)
)
.subscribe((e) => {
this.onScroll(e);
});
}
But this doesn't seems to work and throw below error
ERROR TypeError: Invalid event target
at setupSubscription (fromEvent.js:50)
at Observable._subscribe (fromEvent.js:24)
I also tried moving the code to ngAfterViewInit() but still same error.
I think this issue is growing too much. Devs are reporting here all bugs/questions.
@mmalerba is it fine? or can we use a better strategy?
Like closing this issue and let a message at the end to encourage devs open specific issues/questions... with in the tittle virtual-scroll: since i saw this pattern in all issues from the board you posted.
And maybe also having a specific ReadMe link under 'virtual-scroll' feature folder that document the virtual-scroll and giving a status about the feature.
I'm planning to leave this issue open until the virtual scroll is available in cdk (rather than cdk-experimental). If people have specific bugs they've encountered they should open a new issue. Discussion and troubleshooting should happen on Gitter, Stack Overflow, etc
I wrote a library for it in angualr 6. it uses cdk virtual component and supports end triggers.
check out: https://github.com/aprola/ngx-infinite-virtual-scroll
demo: https://aprola.github.io/ngx-infinite-virtual-scroll/
And Readme.md for instructions. If anyone likes it, ill would love to make documentation too.
@gshireesh Your demo is not working
@manju-reddys thanks for pointing it out. I busted npm cache. now it works fine.
Do you think that this features could be used for virtualization of horizontal scrolling?
@mmalerba Is there ongoing work being done on this AutosizeVirtualScrollStrategy? It looks like the code has been stagnant for 3 months.
Any news on autosize?
This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.
Read more about our automatic conversation locking policy.
This action has been performed automatically by a bot.