Items not showing when using larger data
Closed this issue · 9 comments
Hello I am using version 2.5.0 of this library and 14.8.1 version of vaadin.
Combobox is loaded with 10000 strings:
final List<String> a = IntStream.range(0, 10000).mapToObj(String::valueOf).toList();
MultiselectComboBox<String> location = new MultiselectComboBox<>();
location.setItems(a);
add(location);
When Combobox is opened and scrolled down using scrollbar fast to end items do not show and these messages appear:
2022-01-12 17:03:41.179 WARN 41427 --- [nio-8080-exec-1] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '5800', maximum items allowed '1000'.
2022-01-12 17:03:41.203 WARN 41427 --- [io-8080-exec-10] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '7400', maximum items allowed '1000'.
2022-01-12 17:03:41.204 WARN 41427 --- [io-8080-exec-10] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '7450', maximum items allowed '1000'.
2022-01-12 17:03:41.235 WARN 41427 --- [nio-8080-exec-5] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '9850', maximum items allowed '1000'.
2022-01-12 17:03:41.253 WARN 41427 --- [nio-8080-exec-4] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '10000', maximum items allowed '1000'.
Please help
Is there already a solution or a workaround?
I am just using older version of Vaadin for now.
Absolut urgent also for us.
This issue also exists in 4.0.0-rc1 and showstopped us to use the new Version
@gatanaso can you look at this. Other checks until now are fine
Seems to be that these lines are the problem
Hi @osenbg, if you are using 4.0.0-rc1 with Vaadin 22+, then you can specify a larger page size (default is 50) when initializing the component, for example:
MultiselectComboBox<String> location = new MultiselectComboBox(1000);
This will ensure the requested pages are not out of the valid range.
Hi Goran,
thanks for the hint.
We will try this
Stefan
@zima91 After investigating, it appears the issue might lie in the DataCommunicator class in flow-data-2.7.7.jar from Vaadin 14.8.1 which looks like:
/**
* Sets the requested range of data to be sent.
*
* @param start
* the start of the requested range
* @param length
* the end of the requested range
*/
public void setRequestedRange(int start, int length) {
if (length > MAXIMUM_ALLOWED_ITEMS) {
getLogger().warn(
"Attempted to fetch more items from server than allowed "
+ "in one go: number of items requested '{}', maximum "
+ "items allowed '{}'.",
length, MAXIMUM_ALLOWED_ITEMS);
}
requestedRange = Range.withLength(start,
Math.min(length, MAXIMUM_ALLOWED_ITEMS));
requestFlush();
}
And the MAXIMUM_ALLOWED_ITEMS is set to 1000.
This piece of code looks different from previous versions of flow-data, i.e. from 2.0.12 it looks like:
/**
* Sets the requested range of data to be sent.
*
* @param start
* the start of the requested range
* @param length
* the end of the requested range
*/
public void setRequestedRange(int start, int length) {
requestedRange = Range.withLength(start, length);
requestFlush();
}
And in the latest versions of flow-data it looks like: https://github.com/vaadin/flow/blob/92e2f5c3e006a3ee8a2e4aad33fe3738855de826/flow-data/src/main/java/com/vaadin/flow/data/provider/DataCommunicator.java#L331. Here we can i.e. set a value for the pageSize and with that effectively increase the requested range (similar to my previous comment).
BR
Hi @gatanaso!
Unfortunately the pageSize workaround doesn't work with the last LTS Vaadin v14 version (14.8.9) because as You already mentioned it has MAXIMUM_ALLOWED_ITEMS set to 1000, and not depends on pageSize.
Right now when we try to display more than 1000 elements in MultiselectComboBox with slow or fast scrolling to the end, the Attempted to fetch more items from server than allowed in one go: number of items requested '...', maximum items allowed '1000'. message appears in the log, and the spinner is spinning endlessly on the top right corner of the component. The already fetched elements are visible, but the new ones are not loading anymore, their place is there, but the "lines" are empty.
Tried with org.vaadin.gatanaso:multiselect-combo-box-flow 2.5.0 and 3.0.3 with the following code, but no good (used the official Vaadin 14 – Spring Boot starter for a new, clean project).
package org.vaadin.example;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import org.apache.commons.lang3.StringUtils;
import org.vaadin.gatanaso.MultiselectComboBox;
import com.vaadin.flow.component.combobox.ComboBox;
import com.vaadin.flow.component.dependency.CssImport;
import com.vaadin.flow.component.orderedlayout.VerticalLayout;
import com.vaadin.flow.router.Route;
@Route
@CssImport("./styles/shared-styles.css")
@CssImport(value = "./styles/vaadin-text-field-styles.css", themeFor = "vaadin-text-field")
public class MainView extends VerticalLayout {
public MainView() {
final List<String> items = IntStream.range(0, 10001).mapToObj(i -> StringUtils.leftPad(String.valueOf(i), 5, '0')).collect(Collectors.toList());
MultiselectComboBox<String> mscb = new MultiselectComboBox<>();
mscb.setItems(items);
add(mscb);
ComboBox<String> cb = new ComboBox<>();
cb.setItems(items);
add(cb);
}
}
With 4.0.0-rc2 and Vaadin 23.0.7 and big enough pageSize it works.
The factory Vaadin ComboBox works well in all cases, querying and displaying only the requested elements as we scroll the list, while MultiselectComboBox tries to request a lot more from the DataCommunicator (according to the log warnings anyway).
After that I tried to understand what is happening so I used this code to test the fetching of data:
package org.vaadin.example;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import org.apache.commons.lang3.StringUtils;
import org.vaadin.gatanaso.MultiselectComboBox;
import com.vaadin.flow.component.combobox.ComboBox;
import com.vaadin.flow.component.dependency.CssImport;
import com.vaadin.flow.component.orderedlayout.VerticalLayout;
import com.vaadin.flow.component.textfield.TextArea;
import com.vaadin.flow.function.SerializableFunction;
import com.vaadin.flow.router.Route;
@Route
@CssImport("./styles/shared-styles.css")
@CssImport(value = "./styles/vaadin-text-field-styles.css", themeFor = "vaadin-text-field")
public class MainView extends VerticalLayout {
public MainView() {
final List<String> items = IntStream.range(0, 10001).mapToObj(i -> StringUtils.leftPad(String.valueOf(i), 5, '0')).collect(Collectors.toList());
final SerializableFunction<String, Integer> countFunction = filter -> Math.toIntExact(items.stream().filter(s -> s.contains(filter)).count());
final TextArea logArea = new TextArea("Log");
logArea.setWidthFull();
logArea.setHeight("20em");
add(logArea);
MultiselectComboBox<String> mscb = new MultiselectComboBox<>(100);
mscb.setLabel("MultiselectComboBox");
mscb.setDataProvider((filter, offset, limit) -> {
logArea.setValue(logArea.getValue() + String.format("Querying MsCB items with offset: %s and limit: %s. The filter is %s%n", offset, limit, filter));
return items.stream()
.filter(s -> s.contains(filter))
.skip(offset)
.limit(limit);
}, countFunction);
add(mscb);
ComboBox<String> cb = new ComboBox<>(100);
cb.setLabel("ComboBox");
cb.setDataProvider((filter, offset, limit) -> {
logArea.setValue(logArea.getValue() + String.format("Querying CB items with offset: %s and limit: %s. The filter is %s%n", offset, limit, filter));
return items.stream()
.filter(s -> s.contains(filter))
.skip(offset)
.limit(limit);
}, countFunction);
add(cb);
}
}
Loaded the page, and scrolled down slowly in the MultiselectComboBox. Everything was fine in the log TextArea
Querying MsCB items with offset: 0 and limit: 100. The filter is
Querying MsCB items with offset: 100 and limit: 100. The filter is
Querying MsCB items with offset: 200 and limit: 100. The filter is
Querying MsCB items with offset: 300 and limit: 100. The filter is
Querying MsCB items with offset: 400 and limit: 100. The filter is
Querying MsCB items with offset: 500 and limit: 100. The filter is
Querying MsCB items with offset: 600 and limit: 100. The filter is
Querying MsCB items with offset: 700 and limit: 100. The filter is
Querying MsCB items with offset: 800 and limit: 100. The filter is
Querying MsCB items with offset: 900 and limit: 100. The filter is
until I reached the 1000th element. After that no new TextArea log lines appeared just WARN-s in the console log like these:
2022-04-29 08:44:33.463 WARN 17456 --- [nio-8080-exec-4] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '1100', maximum items allowed '1000'.
2022-04-29 08:49:04.169 WARN 17456 --- [io-8080-exec-10] c.v.flow.data.provider.DataCommunicator : Attempted to fetch more items from server than allowed in one go: number of items requested '1100', maximum items allowed '1000'.
I don't really understand what's going on in the background but the query offset + limit handling seems fine to me before the dreaded 1000th element.
Could you take a look at is please? What should we do if we can't switch to Vaadin v23 and we also can't remain on Vaadin 14.7.x? Thanks.
This is the exact situation we're in right now.
I see that the setRequestedRange method gets called in multiselectComboBoxConnector.js with the start parameter always on 0:
if (cache[params.page]) {
// This may happen after skipping pages by scrolling fast
commitPage(params.page, callback);
} else {
const upperLimit = params.pageSize * (params.page + 1);
if (filterChanged) {
this._debouncer = Debouncer.debounce(
this._debouncer,
timeOut.after(500),
() => {
multiselectComboBox.$server.setRequestedRange(0, upperLimit, params.filter);
if (params.filter === '') {
// Fixes the case when the filter changes
// from '' to something else and back to ''
// within debounce timeout, and the
// DataCommunicator thinks it doesn't need to send data
multiselectComboBox.$server.resetDataCommunicator();
}
});
} else {
multiselectComboBox.$server.setRequestedRange(0, upperLimit, params.filter);
}
pageCallbacks[params.page] = callback;
}Shouldn't the start parameter be the offset and length the page size?