Output mirroring and complex layout configurations
Opened this issue · 57 comments
First step is to make these two statements true:
- Outputs always render the tiled views in their own sway_container and never anyone else's
- Floating views are rendered with respect to the output layout, and can show on multiple outputs
Then, we should start to introduce two new kinds of configuration to each output config:
- Layout configuration
- Mirroring configuration
A layout configuration stores how much of the output's space is allocated to its sway_container:
output HDMI-A-1 layout 100x100@10,10
Bonus: could subdivide the output into multiple layouts
output HDMI-A-1 res 1920x1080 layout 960x1080@0,0 layout 960x1080@960,0 subdivides a 1080p output into two halves, which each get their own workspace (useful for ultrawides, for example)
If no layout is specified, the default layout is used - WxH@0,0
No layout at all can also be specified via layout none, in which case no workspace is allocated.
Separately, you can specify a mirroring configuration via mirror [source output] [at X,Y], which will draw tiling windows from the source output onto the configured output at the target position. You can specify this several times.
Some example configurations:
Normal configuration
output HDMI-A-1 res 1920x1080
output DP-1 res 1920x1080 pos 1920,0
Basic mirroring
output HDMI-A-1 res 1920x1080
output DP-1 layout none mirror HDMI-A-1
Disjoint resolution mirroring
output DP-1 res 1280x720
output HDMI-A-1 res 1920x1080 layout none mirror DP-1
Disjoint resolution mirroring with floating views
Note: this is the same config as the last example
output DP-1 res 1280x720
output HDMI-A-1 res 1920x1080 layout none mirror DP-1
Disjoint resolution mirroring with multiple layouts
output DP-1 res 1280x720
output HDMI-A-1 res 1920x1080 mirror DP-1
Disjoint resolution mirroring with multiple layouts and double mirroring
output DP-1 res 1280x720 mirror HDMI-A-1
output HDMI-A-1 res 1920x1080 mirror DP-1
Partial mirroring plus partial layout
output HDMI-A-1 res 1920x1080
output DP-1 res 1920x1080 mirror HDMI-A-1 at -1800,0 layout 1800x1080@120,0
Unanswered questions:
- z-ordering config
wlr-output-layout has no support for z-ordering so we'll have to address it there.
I don't think that's in scope for wlr-output-layout, we'll probably want to store that info on sway's structures
do you have a roadmap for this issue already?
We never have ETAs.
This message is for people following this ticket and looking for a workaround for the current lack of screen mirroring for instance to use an overhead projector.
The trick is that the mouse and the floating windows are rendered according to their global coordinates.
Also sway does not complain when outputs have overlapping geometries so you can do something like
swaymsg output eDP-1 pos 0 0
swaymsg output HDMI-A-2 pos 0 0
At first that does not seem to do anything except that the mouse is now displayed on both screens.
What is interesting is that all floating windows are now also rendered on both screens.
I am pretty sure that during my first experiments fullscreen windows were rendered on both screens but unfortunately I cannot reproduce that anymore (another trick needed)?
That setup is not without issues. The main one is that sway is tracking the mouse coordinates to between switch between the output workspaces. Consequently, unexpected and mostly unnoticeable switches between the 2 overlapping workspaces can happen during common actions using the mouse (e.g. moving, resizing, leaving or entering floating windows).
I'd prefer a looser coupling between layouts and outputs. Instead of having workspaces on outputs, you have them on layouts. Layouts can have an arbitrary position and size (but probably should not intersect). Outputs can also be arbitrarily positioned. They act as viewports into the layout plane instead of directly having workspaces. To make an output into a viewport (i.e. have no workspaces itself), the output configuration line gets the 'viewport' keyword.
If an output happens to view a region where some layouts lie it'll be a viewport for them. Dunno how DPI scaling of outputs/viewports could be handled.
Example:
layout 100x100 pos 0,0
layout 100x100 pos 100,0
layout 100x100 pos 0,100
layout 100x100 pos 100,100
output DP-1 res 100x100 pos 50,50 viewport
The black rectangles are layouts, the red one is a output/viewport. In this (unrealistic) example you'd see a corner of each of the four layouts on output DP-1.
// Mirrorring
+1 for a config allowing you to mirror any output ... ideally be able to switch with keyboard shortcut to mirror-mode, because my default should be:
// Stacking on top
My setup is 99.9% of all cases: Laptop screen + 1 external screen / Beamer.
@ddevault : With a notebook in-front of a desktop monitor, I would love to extend the external screen above my laptop screen. And have a generic config, that by default stacks the external screen on-top of the laptop screen.
It seems to me that with the current sway-output syntax this should not be possible, because the 0|0 coordinate is in the top left corner and thus I would need to use the height of the external screen in the sway config... but this height can vary depending on screen model... and I am looking for a way to use the "*" to match any external screen.
I hope mirroring comes soon, that is the missing feature that forced me back to X11 :-(
Would this issue also cover handling different output configurations with different combinations of connected displays?
e.g. if I am using just my laptop screen, then that should be turned on
if I have connected my laptop to my external monitor, then the external display should be on and the laptop screen should be off.
There is currently no way to declare this (from what I can tell) in sway configuration.
There are no plans to support such thing in sway. You can use a helper client instead, such as kanshi.
Was not aware of kanshis existance. Thanks for the heads up @emersion :)
Is there any progress on this? Honestly speaking, not being able to switch between dual and mirror mode is a major problem for most laptop users to the point that they can't use sway at all.
Is there any progress on this? Honestly speaking, not being able to switch between dual and mirror mode is a major problem for most laptop users to the point that they can't use sway at all.
I'm not sure about not being able to use sway at all, but I agree it would be nice to at least have mirroring work. It is a bit of a bummer to have to switch back to i3 for conferences when I have to project the screen for audiences.
Would it be better to split mirroring off as a sub-feature and implement instead of waiting for this whole feature set to be done together?
Some hint for conferences and presentations: I was able to make OBS Studio (with wlrobs) to display a window with real time contents of another display (big screen/projector) which I can view on laptop display which is quite enough for presentations. I suppose there should be more lightweight solution but I haven't got enough time to dig into it.
Would it be better to split mirroring off as a sub-feature and implement instead of waiting for this whole feature set to be done together?
Yes. The plan was never to implement everything in this issue at once. Incremental pull requests are welcome.
I see this enhancement is not in a milestone or project.
So I am wondering how far are we with the two first steps mentioned by OP @ddevault
First step is to make these two statements true:
- Outputs always render the tiled views in their own sway_container and never anyone else's
- Floating views are rendered with respect to the output layout, and can show on multiple outputs
in regards to starting work on the following?
Then, we should start to introduce two new kinds of configuration to each output config:
- Layout configuration
- Mirroring configuration
I get the incremental PR approach. Though as an sway dev "outsider", it is hard to tell which state or where in the process this enhancement is. Could you enlighten me?
Thanks
Floating views are rendered with respect to the output layout, and can show on multiple outputs
I just launched both an Xwayland (xeyes) and a wayland native app (alacritty) each, set them to floating, and dragged them between my outputs. They both rendered partially on each output as expected conformal to the defined layout. So at least one of two prerequisites is met.
Is mosaic of displays something that fits here or that should have its own issue?
For @torwag or anyone else that like me come here from a search, I have a non-solution that I settled on for now.
I just run swaymsg output eDP-1 disable to disable the laptop screen so everything is moved to the external monitor, that's what I was trying to achieve with mirroring anyway. You can then swaymsg output eDP-1 enable to enable it back.
There's also an entry on the sway wiki about doing it automatically when lid is closed: https://github.com/swaywm/sway/wiki#clamshell-mode
Hope someone finds this helpful.
@tasn Indeed this is not a solution at all. Sway does manage very well extended workspace across multiple screens. What is missing is a mirroring mode to be able to see what is being presented when the secondary screen is behind you for instance. Your solution does not change anything on that subject.
I wasn't trying to claim that it does. I was just suggesting that for the use-case of plugging your laptop into a dock, there's a workaround until this is implemented.
Currently you can run wayvnc server on one screen and use a vnc client to display that on another screen. it works somewhat better and is usable for slideshows.
The requirement for mirroring for me is to do presentations and meetings. Often you don't see the big screen at all, so you need to be able to look at your laptop screen in order to see what everyone else is seeing. Sway is great, but I've had to move back to i3wm just to support mirroring.
Many presentation applications support "presentation mode" which has one view that you can put on the big screen, and one just for you with notes + timer. I always use that when giving talks.
I agree mirroring would be useful, just trying to help with solutions for some common use-cases, at least the ones I encounter. :)
Back when I switched to sway from i3 I did this willingly to take the pain of the missing output mirroring. Then it turned out such pain never materialized, as almost every program you are likely to want to use for presenting offers some sort of dual-screen presentation mode. I mostly use my own, but for my occasional PDF slides I discovered by accident that Okular does it too: Start the presentation with Cntrl-Shift-p and move the full-screen slide to the beamer with Win-Shift-2 (or whatever). The regular Okular window reappears behind the moved audience view; you can navigate its pages, and the audience view will happily follow in sync.
Last Friday I gave a full-day course using Jupyter Lab. For the first time I was really going to miss output mirroring, so I thought, which brought me back here. The suggestion of using VNC struck me as such a kludge!
I installed wayvnc and gave my course, using vinagre to remote-control the audience view.
Now, output mirroring is such a kludge! It forces you to sacrifice your entire internal screen to the audience. VNC does exactly the right thing: It displays the audience view in a window that you can move, resize, hide, etc., giving you full control over the presentation without getting in the way.
I no longer see any use case for output mirroring.
Edit: In the meantime, wl-mirror has appeared on the scene. This does exactly the right thing and is much more lightweight than VNC.
Output mirroring, or at least mirroring of the contents of a sway "container" is something that is definitely something very needed outside of just presenting a single application.
Doing hands-on courses or presentations often means you have to show multiple applications side-by-side (e.g. terminal and browser), and at least until screen recording and streaming works well on wayland you can't even outsource this to an application (e.g. obs) in a simple way. And even if such applications exist, the simplicity of just saying "mirror" will deter MANY people from switching to sway if they use it regularly and sway doesn't support it.
On a different note: Complex Layout configurations
Ultrawide and 4K screens are becoming more popular, and many people that had multiple screens previously are switching or thinking about switching to a single larger monitor. In such situations complex layout configurations like splitting a single monitor into two or more logical workspaces is a MUST. It's pretty much unusable otherwise.
I tried to read into the sway and wlroots code base multiple times, and even tried to ask on IRC for some documentation, but the answer was mostly "implement it yourself and send a patch", and "just read the code".
Ultrawide and 4K screens are becoming more popular, and many people that had multiple screens previously are switching or thinking about switching to a single larger monitor. In such situations complex layout configurations like splitting a single monitor into two or more logical workspaces is a MUST.
That's exactly why I'm interested in this.
Output mirroring, or at least mirroring of the contents of a sway "container" is something that is definitely something very needed outside of just presenting a single application.
Doing hands-on courses or presentations often means you have to show multiple applications side-by-side (e.g. terminal and browser), and at least until screen recording and streaming works well on wayland you can't even outsource this to an application (e.g. obs) in a simple way. And even if such applications exist, the simplicity of just saying "mirror" will deter MANY people from switching to sway if they use it regularly and sway doesn't support it.
On a different note: Complex Layout configurations
Ultrawide and 4K screens are becoming more popular, and many people that had multiple screens previously are switching or thinking about switching to a single larger monitor. In such situations complex layout configurations like splitting a single monitor into two or more logical workspaces is a MUST. It's pretty much unusable otherwise.
Why was THIS comment marked as "off-topic"? It was essentially showing the uses of output mirroring and complex layout configurations, and situations in which it ia useful, which is
a) exactly the kind of thing this issue is about
b) a semi-direct rebuttal to @piater's comment saying there is no need for output mirroring
I'd have found it understandable if my second remark about the state of documentation in this project would have been marked off-topic instead, since it arguably is, but it was exactly the other way round.
Would it make sense to break this issue into two separate issues? I'm noticing that so far, a lot of the conversation here has been specifically about the usefulness or non-usefulness of screen mirroring, but I've actually been following this thread for the use case of dividing up larger monitors into several discrete workspaces.
If the underlying technical work is the same, having them in the same issue makes sense. It just seems like there's a risk that the conversations about mirroring utility might lead people to pass over this feature request, when there is actually a more broad request for advanced layout configurations, that may also be used for mirroring if someone needs it.
b) a semi-direct rebuttal to @piater's comment saying there is no need for output mirroring
@Ferdi265, this is not what I said and not what you rebutted. First, I talked about my perception; YMMV of course. (At least I meant to talk about my perception; I see now that my wording was a bit strong.)
Secondly, I don't want my output (screen) to be mirrored because this prevents me from using my internal screen for other purposes. However,
mirroring of the contents of a sway "container"
is a fantastic idea! This would allow you to interact directly with Sway windows as opposed to manipulating windows inside the VNC window, while keeping your internal screen available for other purposes as well.
Moreover, if we had a way to divide an output into multiple workspaces, then you would not even have to alternate between them, you could change their size, you could "full-screen" applications without them taking up the entire physical screen, etc. How cool would that be!
Crucially, I prefer to think of the external output as being mirrored into a container (or, at present, VNC client window) on my internal screen, not the other way around. For example, as I resize my internal view, the external view remains unchanged at its native resolution.
Locking this conversation due to too many off-topic comments. If you're interested in working on one of the features describes here, please ping us on IRC.
swaywm/wlroots#3067 might come in handy.
Is there any way to mirror display to, for example, a projector?
I would actually need that next week.
Is there another app that could make this work?
@ShinobiX9X wl-mirror does the trick quite well.
I'd prefer a looser coupling between layouts and outputs. Instead of having workspaces on outputs, you have them on layouts. Layouts can have an arbitrary position and size (but probably should not intersect). Outputs can also be arbitrarily positioned. They act as viewports into the layout plane instead of directly having workspaces. To make an output into a viewport (i.e. have no workspaces itself), the output configuration line gets the 'viewport' keyword. If an output happens to view a region where some layouts lie it'll be a viewport for them. Dunno how DPI scaling of outputs/viewports could be handled.
Example:
layout 100x100 pos 0,0 layout 100x100 pos 100,0 layout 100x100 pos 0,100 layout 100x100 pos 100,100 output DP-1 res 100x100 pos 50,50 viewport
Did this got anywhere? I have a real world example where this would be useful.
Ultrawide monitor split into 3 virtual monitors. Can that be achieved in any way?
That's exactly what I had in mind when I wrote that. A 21:9 monitor works okay without that,, though on a 32:9 I briefly had it was really necessary. @emersion did write a patch that does in the other direction (mentioned above), but other than that nothing has happened. I'd still be interested in seeing it happen though, since I think the concept of decoupling is useful in general.
On a different note: Complex Layout configurations
Ultrawide and 4K screens are becoming more popular, and many people that had multiple screens previously are switching or thinking about switching to a single larger monitor. In such situations complex layout configurations like splitting a single monitor into two or more logical workspaces is a MUST. It's pretty much unusable otherwise.
Note: I hope this comment doesn't get categorized as off-topic. My goal is to be helpful to future people trying to search for this feature.
I've searched for this several times but this is the first time I've found an issue discussing this feature. Every time in the past I've tried to search for fake-outputs which is what the feature is called in i3. I've held off on switching to sway for a while because I'd like to use that same fake-outputs functionality (I have 42" TV as a monitor, split virtually into 3 spaces).
Wouldn't it make sense to split this issue? The "mirror" feature is probably very easy to implement right?
Wouldn't it make sense to split this issue? The "mirror" feature is probably very easy to implement or?
Just to reiterate from the thread: https://github.com/Ferdi265/wl-mirror
I think this is light years better than "classic" mirroring. So maybe the question could be: Wouldn't it make sense to move this into sway (though I am all for the add-on/package/plugin structure)
Because other comments suggesting wl-mirror have received some down-votes and because I myself had no clue why wl-mirrors is "light years better" than "classic" mirroring or vnc based solutions, I tried it out myself and though that maybe what I found out could be useful to others that look for a mirroring solution.
I'm running sway on an ARM Cortex A53 at 1.5 GHz. This means my platform is very slow compared to modern intel machines. Browsing websites that heavily make use of JavaScript is no fun, for example. So I was hesitant to try and use wl-mirror.
But what I found was: wl-mirror does not even use 1% of my CPU even while recursively mirror-ing itself. Even when I'm playing a video at 1080p I do not observe any CPU spikes nor any perceivable lag before all the nested copies get updated. How is this possible? To quote the author from https://bugs.debian.org/1031284:
wl-mirror by default uses the wlr-export-dmabuf protocol extension to receive frames from the compositor. The compositor hands wl-mirror a dmabuf handle, which is a handle to frame data already in VRAM
on the GPU. Importing it into EGL and rendering to the screen therefore does not include copying it to the CPU and back, which is why the CPU overhead does not depend that much on screen resolution and
activity in the image.
I like the UNIX philosophy of doing one thing and doing it well. Since wl-mirror already is doing a perfect job with no lag and nearly no CPU usage, I do not see why sway would need to have that feature added as well.
Because it is not user friendly. A normal Sway user knows nothing about wl-mirror!
I do not see why sway would need to have that feature added as well.
Because this feature is not only about mirroring, and wl-mirror cannot split an ultrawide monitor in two workspaces.
Because it is not user friendly. A normal Sway user knows nothing about wl-mirror!
Then sway could document or recommend it better so that it's easier to find for sway users.
Sway doesn't have to do everything. The systems we use are not monolithic but a collection of different tools doing different things. I also do not argue that the cat tool should add the ability to filter the data it prints just because I've never heard of the grep utility.
Because this feature is not only about mirroring, and wl-mirror cannot split an ultrawide monitor in two workspaces.
True. This problem (as stated above already) is that this issue is about two issues. My comment about wl-mirror only addressed the "output mirroring" part and not the "split ultrawide mirror in two workspaces" part.
Then sway could document or recommend it better so that it's easier to find for sway users.
I agree. Even better: directly as wrapper command: "swaymsg output DP-1 mirror wl-mirror".
I did not down-vote wl-mirror but I do not think that the main issue is the CPU usage. IMHO, this is that wl-mirror is using its own workspace. I tried to use it last year for a simple presentation and that was not an enjoyable experience. It is very easy to become confused while switching workspaces if you accidentally select the workspace running wl-mirror (by moving the mouse or with the keyboard). The last thing I want during a presentation is to start fighting against my desktop.
The problem I am describing here probably has some solutions. I did not look into it but it should be possible to change my sway config so that the mirror output and workspace cannot be (easily) switched to. A possible and simple solution could be a sway command to make a workspace unfocusable.
Given the current state and trajectory of X11 I'm wondering if there's anything an outsider (current non-contributor to this repository) can do to help this along? I'm specifically looking for the functionality described in #1666 (comment) so that I can switch from i3 to sway.
Is there a bug (feature) bounty ($$) we could start that would help?
In a previous lifetime where I was more adventurous and had copious amounts of free time I would have attempted to make the contribution myself, but that's not my reality today.
Due to the lack of this feature I had to buy two monitors instead of the ultra wide one and use black electrical tape to hold them together (not on the table). Maybe I should buy better onkron but anyway this feature is very desirable :)
Is there a bug (feature) bounty (
$$) we could start that would help?
I strongly second this. I do not have the time or skills to try to implement this feature for such a used project, but I would be happy to contribute by throwing in a few bucks. If different people contribute, we may attract developers to implement the feature, and make everybody happy!
Is there a bug (feature) bounty (
$$) we could start that would help?I strongly second this. I do not have the time or skills to try to implement this feature for such a used project, but I would be happy to contribute by throwing in a few bucks. If different people contribute, we may attract developers to implement the feature, and make everybody happy!
I give a talk tomorrow which reminds me that I still won't have output mirroring. This is becoming seriously embarrassing. Can we at least get some approval from the dev. team to launch a bounty? It would be better and more right to do that with you guys on board; we are all aware that the result will be better if you are involved and have control over the implementation. But it has been one year since @dustymabe's first mention of a bounty, and three months since mine. Sway is probably the most widely used tiling WM on Wayland, and this feature (independently from its difficulty to implement) would be considered basic in any DE...
With love,
Kryzar
Sorry, we don't do bounties.
Thanks for answering. May I ask why? What would be the way to have the feature implemented? Many want it, few can do it with quality. Also, does that mean that you would refuse a PR if it originated from a bounty?
Bounties come with a bunch of drawbacks and potentially toxic consequences, for instance competing contributors or frustrated contributors if a patch needs many review iterations.
That's why we suggest to do that with you on board (if well organized and with clear rules, those problems are easily manageable; the consequences you mention are very general and do not apply to all situations). The best would even be to pay you with the funding from bounty; I'm sure many would be willing to do so.
However, if you really insist on not organizing that, which we respect, could you answer to the two other questions?
What would be the way to have the feature implemented?
Also, does that mean that you would refuse a PR if it originated from a bounty?
I am well aware that this is FOSS and you don't owe anyone anything. However, the WM is a big part of the desktop experience of many people, and I think those are entitled of at least knowing if the feature is on the roadmap. If you say that you do not intend to implement the feature for the next X years, that's fine, but I believe people would likely want to know, and change their config if necessary.
Thank you very much for your time and consideration (and Sway!).
The best would even be to pay you with the funding from bounty
Indeed. My use of the term "bug bounty" may have not been clear. I definitely don't want users/contributors competing for the bounty. More clear would be either paying @emersion (or some other known contributor) directly for the work somehow. Use of a bug bounty like platform could enable us to transfer those funds.