Improving layouting

Question

Improving layouting

emilk opened this issue 5 months ago · 6 comments

Understanding the problem

One of the main drawback of single-pass immediate mode GUIs is layouting. There is a cyclical dependency between position, size, and layout. This can be illustrated explained by an example:

Say you want to show a small dialog window in the center of the screen. To position the window we must first know the size of it. To know the size of it we must first layout the contents of the window. In immediate mode, the layout code also checks for interaction ("was the OK button clicked?"), thus we need to know where to place the OK button before doing the layout. So we need to know where to put the window before we can layout the contents of the window, but only after laying out the contents of the window can we know the size of the window, and thus where to place it so that it is centered on the screen.

The crux of the issue is really knowing the size of a thing before you can lay it out properly. So, how can we know the size of a thing before calling the layout code for it?

There are simple atomic widgets like egui::Button that calculates its size based on the text it contains and the border. This is where egui "bottoms out". These atomic widgets can be properly centered, because they only result in a single call to ui.allocate*.

Whenever you have a group of widgets though, you have a problem.

Figuring out sizes before layout

So how can we know the size of a thing before doing the layout?

There are a few different ways:

A) Using fixed size elements (e.g. decide that the dialog window is always exactly 300x200 no matter what it contains)
B) Remember sizes of elements from previous frames (first frame calculate sizes, all other frames: position things based on those sizes). egui uses this strategy for some stuff (like Window, Grid, Table, …)
C) An API for calculating the size of an element before adding it (#606 - has potential for exponential slowdowns)
D) Multi-pass immediate mode (rejected)

I think B) is the most interesting one to dig deeper into.

Whenever the B) strategy is used, one should be careful not to show the widget during the "layout" frame. For instance, a centered egui::Window is invisible for the first frame.

The B) strategy also fails if the thing changes size over time. Though it is self correcting, it has a frame-delay. So in the original example: if the dialog windows grows it will shift to the right for one frame, then re-center then next frame. This will look ugly.

Improving layout given sizes

Once you have the size you should be able to apply any advanced layouting technique. For instance using:

Here we have a lot of opportunity for improvement in egui. As a start, we should at least write good examples for all of these.

Answer 1 · 2024-04-18T16:54:59.000Z

I think some issues like #1996 #2786 #2798 #3054 #4159 are related to this.

Answer 2 · 2024-04-27T01:44:19.000Z

I recently also ran into this while trying to fix #3074. One other possible way, at least where it's an issue of translations, is to fake everything being fine by fixing it afterwards. This should work when it's happening while the mouse is already occupied doing, like dragging windows around, since there wouldn't be any way to notice the response being 1 frame delayed. Unfortunately it falls apart for anything more complex, or when it is possible to do two things at once like with touch controls, or when there are massive differences between the estimated and actual positions.

That does beg the question, how is correct interactivity vs visuals weighed? I think more people will notice when visuals are delayed by a frame, while noticing a lagging response is harder due to them being invisible. Is that an acceptable tradeoff, and what other methods like translations and hiding windows for an initial frame are available when it is?

Edit 2: Leaving some of this here but removing a lot of my rambling. The more I think on this, the more it's turning into a worse version of multi-pass immediate mode, so probably the wrong direction if not applied carefully.

Answer 3 · 2024-05-11T23:19:06.000Z

Perhaps a mix of immedieate and pre layouted mode would be a good idea? Perhaps implement a widget that acts as a container for pre layouted widgets. Those widgets still get drawn on the screen every frame, but cannot change their bounding box during a frame. These pre layouted widgets could request a resize to be performed before the next redraw. You could still do things like change color on hover or highlight a button this way during draw.

For those pre layouted containers you could implement some more advanced layouts aswell as provide a trait users could implement to provide their own layouting.

A big con is however these pre layouted components would probably require a completely different api. Sizing information would probably need to be made available in a dedicated method. Id especially pay attention to text sizes as those are a bitch in every other framework that does pre layouting. The draw method would need to be told its bounding box and perhaps steps should even be taken to prevent drawing outside of the bounding box.

Adding/Useing immedeate widgets inside of pre layouted components/Containers should be possible without much issue if you assign a bounding box with a size known to the layouter to them.

I think many people will fine pre layouting for some use cases more intuitive than others. Allowing users to explicitly combine them with immedieate widgets would probably be ideal, at least in my opinion.

Answer 4 · 2024-06-06T12:02:14.000Z

I experimented a bit with a new Group container which uses the new sizing pass to calculate the size the first frame, and then store it for later. This lets you put a group of widgets in e.g. a centered layout

#4631

Answer 5 · 2024-08-16T02:42:50.000Z

Item B is a very close match for how React useRef hooks work, and the problems solved by 'react hooks' in general are a close match for various issues found in immediate mode UI rendering.

Answer 6 · 2024-09-06T10:55:50.000Z

I've been experimenting with improving layout in egui for a while, some time back I made egui_taffy which works pretty well but it didn't feel nice to use with egui, since it had to borrow the ui closures for until the taffy pass was finished which caused some lifetime issues, so I've never finished or published it.

introducing egui_flex

I've been experimenting how much of flexbox I could implement in egui by just remembering the widget sizes from the previous frame and the results are quite impressive! I've published them here: egui_flex

First of all, here is a good refresher what all the different flex keywords mean.

The following things work as expected:

flex-direction: row and column work as expected (I've named them horizontal and vertical to match egui's layout names)
flex-grow: you can give items a grow factor and they will grow to exactly fill the row/column. An item with grow: 2 will grow twice as much as an item with grow: 1
align-items / align-self:
align-items-content / align-self-content: egui-specific property I added to help align an item's content if it has grow > 0.0 || align_self == Stretch
nested flex containers work and will grow as expected as long as they don't wrap:

Every group in the screenshot represents a flex container or flex item so this shows flexes nested three levels deep.

You can't arbitrarily nest flexes in child ui's, you have to use a special method on the flex builder to add nested flexes, because it needs to communicate it's minimal size to the parent flex.

The following things aren't implemented yet but should be possible:

justify-content: should be easy to add
handling wrapping in nested flex: not 100% certain but I think this should be possible

The following things would require a modification in egui

egui Buttons currently can't grow their frame when added as a flex item. In order to do that, there needs to be some way to add a button with a certain size while getting it's intrinsic size. One way to do that could be by adding a intrinsic_size field to the Response struct (in case of the button that would be set to galley.size + button_padding (I think adding this could be handled in allocate_at_least based on the desired_size))

As a workaround I've added a FlexButton and a FlexWidget trait. The button is just a reimplementation of the default button that allows reporting the intrinsic size while growing it's frame.

The following things are not possible

flex shrink:
Getting a items intrinsic size requires adding the item without limiting it's size and to shrink an item we must limit it's size.
Shrinking an item with a fixed size should in theory be possible.
flex-wrap: no-wrap
Since we can't shrink items there is no point in having flex-wrap: no-wrap (so we always wrap)

Here's a demo showing how nice things flow into the next row when resizing the window:

Bildschirmaufnahme.2024-09-06.um.12.52.10.mov

Real world example

Finally I wanted to share a real world example how flex can improve the ui by a lot. In the hello_egui demo app I have a list of crates displayed as small tags in the sidebar. When just shown with ui.horizontal_wrapped it looks really weird:

When updated to use egui_flex it looks much nicer:

egui or separate crate?

I plan on publishing this as a separate crate but I think it could also make sense to add this to egui similar to the Grid layout?
Maybe it could even be the default layout in the future? I think it should be pretty compatible with egui's current horizontal / vertical / with_layout layouting while solving most problems it currently has. @emilk what are your thoughts on this?