[Question] Virtualizing Recommendations

[nickolaschong]

Hi Norigin Media Team,

I would like to take a moment to express my sincere gratitude for your team contribution to the open source community through the Norigin-Spatial-Navigation package. I truly appreciate the time and effort your team have put into developing and maintaining this package. The dedication to creating high-quality open source software is truly admirable and passing on a positive impact on the TV developer space.

Question: I would like to know the team's opinion on whether there are any recommends package for 'virtualizing' large number of rails, horizontal and vertically to improve memory management.
We have integrate with https://github.com/bvaughn/react-window, however it will does not work as elegantly / fuzz free as we it wanted to.

[krackjack234]

Hi Norigin Media team,

Further to the question posted above, we have put together a sample app to show the problem
We used the react window library for virtualisation and used norigin spatial navigation library for remote key navigation.

Here is what we tried

1. Call a paginated API to load 20 items at a time and render in a list
2. Set initial focus on the last item
3. Scroll left
4. Once we reach the end of the list, call the paginated api to load 20 more items from the previous page and append them to the beginning of the list

Sample code is shared here

Problem

If we keep on scrolling to the left and keep on loading new data, the focus suddenly shifts to the first element on pressing the Up button

Logs

Looking at the logs it looks like the norigin is calculating the siblings wrongly and hence jumping to the wrong item

index.js:1 setFocusfocusKey Cell-1991
index.js:1 getNextFocusKeytargetFocusKey Cell-1991
index.js:1 setFocusnewFocusKey Cell-1991
index.js:1 saveLastFocusedChildKeyColumn lastFocusedChildKey set Cell-1992
index.js:1 saveLastFocusedChildKeyColumn lastFocusedChildKey set Cell-1992
index.js:1 smartNavigatedirection up
index.js:1 smartNavigatefromParentFocusKey null
index.js:1 smartNavigatethis.focusKey Cell-1991
index.js:1 smartNavigatecurrentComponent Cell-1991 
index.js:1 smartNavigatecurrentCutoffCoordinate 8
index.js:1 smartNavigatesiblings 6 elements: Cell-1981, Cell-1982, Cell-1983, Cell-1984, Cell-1985, Cell-1986 
index.js:1 smartNavigatedistance (primary, secondary, total weighted) for Cell-1981 relative to Cell-1991 is 200 8 1008
index.js:1 smartNavigatepriority for Cell-1981 relative to Cell-1991 is 1009
index.js:1 smartNavigatedistance (primary, secondary, total weighted) for Cell-1982 relative to Cell-1991 is 200 8 1008
index.js:1 smartNavigatepriority for Cell-1982 relative to Cell-1991 is 1009
index.js:1 smartNavigatedistance (primary, secondary, total weighted) for Cell-1983 relative to Cell-1991 is 200 8 1008
index.js:1 smartNavigatepriority for Cell-1983 relative to Cell-1991 is 1009
index.js:1 smartNavigatedistance (primary, secondary, total weighted) for Cell-1984 relative to Cell-1991 is 200 8 1008
index.js:1 smartNavigatepriority for Cell-1984 relative to Cell-1991 is 1009
index.js:1 smartNavigatedistance (primary, secondary, total weighted) for Cell-1985 relative to Cell-1991 is 200 8 1008
index.js:1 smartNavigatepriority for Cell-1985 relative to Cell-1991 is 1009
index.js:1 smartNavigatedistance (primary, secondary, total weighted) for Cell-1986 relative to Cell-1991 is 200 8 1008
index.js:1 smartNavigatepriority for Cell-1986 relative to Cell-1991 is 1009
index.js:1 setFocusfocusKey Cell-1981
index.js:1 getNextFocusKeytargetFocusKey Cell-1981
index.js:1 setFocusnewFocusKey Cell-1981
index.js:1 saveLastFocusedChildKeyColumn lastFocusedChildKey set Cell-1991
Columns.tsx:75 focused index:  0  text:  Ep 1981 
index.js:1 saveLastFocusedChildKeyColumn lastFocusedChildKey set Cell-1991
index.js:1 setFocusfocusKey Cell-1981

Even though Cell-1990 is visually closer to Cell-1991, on pressing the Up button, the focus jumps to Cell-1981 which is the first element of that page.

Case-1.webm

Kindly let us know what can be the possible root cause of this problem.

Thanks.

[kaufmann42]

Having a similar problem here

[Uttu316]

Hey, @nickolaschong @krackjack234 , were you able to figure out a proper solution of windowing using this package?

[nickolaschong]

@Uttu316 No. We have yet attempt to solve horizontal virtualization but we wrote a custom function to solve vertical virtualization. (Pages with unlimited rails vertically)

If anyone else is interested on how we tackled vertical virtualization.

Technique: Combination of Frustum culling + Level of Detail (LOD)

Frustum culling is the technique used in game engines to determine and render only the objects that fall within the camera's view frustum. It involves eliminating or "culling" objects that are outside the view frustum, as they are not visible to the player and do not need to be rendered.

Level of Detail (LOD). LOD is a rendering technique used in computer graphics and game engines to manage the level of detail displayed for objects based on their distance from the viewer.

Once you understand this technique, all you need to is to implement the culling of rails and replace them with a Low level detail element like a Dummy Rail when they are not in the viewport buffer. This frees up tremendous amount of memory on the TV (tested on 2015 devices) while maintaining the correct scroll position.

Step 1: Maximising useSelector hook.
Whenever the rail is in isRailInViewportBuffer then only this component will re-render.

const currentFocusedRailIndex = useSelector((state: any) => {
  const focusedIndex = state.page.cache[location]?.currentFocusedRailIndex || 0;

  if (isRailInViewportBuffer(focusedIndex, railIndex)) {
    return focusedIndex;
  }
});

const inViewPortBuffer = isRailInViewportBuffer(currentFocusedRailIndex, railIndex)

return (
    <>
        {inViewPortBuffer ? (
            // Render actual rails
        ) : (
            // Render dummy rails     
        )}
    </>
);

Step 2: isRailInViewportBuffer.

export const isRailInViewportBuffer = (focusedIndex: number, index: number) => {
    const lowerBound = focusedIndex - RailConfig.bufferSize;
    const upperBound = focusedIndex + RailConfig.bufferSize;
    const inBuffer = index >= lowerBound && index <= upperBound;

    return inBuffer;
};

[predikament]

Closing this issue as it's relatively stale at this point.

From what I can see the solution above is how we approach the same problem, so if anyone else is looking for a solution to this they can see @nickolaschong's comment above. Thanks for the input.