Switch to xterm-style mouse reporting
egmontkob opened this issue · 4 comments
Intro
GPM should switch to reporting mouse events as part of the input stream towards apps, as graphical terminal emulators do.
Why?
GPM mouse support typically only works in apps that run directly in the console. It does not work inside screen or tmux, does not work inside luit, does not work across a su or sudo, does not work over ssh. With xterm-style reporting you get these for free.
Mouse handling apps would no longer need to link against gpm and have a dependency on this package (the library). Their codebase could probably be simplified, they could drop all the gpm-specific code, and retain only the xterm-specific code (which they presumably already have), that one would handle gpm events as well.
Similarly, ncurses could also drop its gpm dependency, eliminating the dependency loop between ncurses and gpm (#19).
Current state
The Linux kernel has supported xterm-style mouse reporting since 2000. The required call is ioctl(..., TIOCLINUX, ...), the first byte of the payload is TIOCL_SETSEL (2), then 4 shorts for the selection's (ouch, what selection?) coordinates, and another short with the TIOCL_SELMOUSEREPORT bit set, in which case TIOCL_SETSEL is in my opinion semantically misused to report a mouse event according to the TIOCL_SELBUTTONMASK mask, rather than defining a selection.
A patch called xterm-mouse-for-console.patch was created also back in 2000. I've found it inside gpm-1.99.7's source tree, but not under gpm-1.20.7's. Probably no need to say, the patch created 18 years ago does not apply.
In #4 @seyko2 created another patch in 2014, that attempt also stalled.
Kernel limitations, desired work
The legacy xterm mouse protocol, supported by the kernel, can only report coordinates up to 223. I think the Linux console can be wider than this (e.g. in framebuffer mode), can't it? Moreover, coordinates above 95 are broken by charset converting layers (e.g. luit). There were two unsuccessful (technically problematic) attempts to fix this (extensions 1005 and 1015), and one apparently successful widespreadly used one (1006). The kernel should add support for the latter.
The implementation of the legacy mode in the kernel suffers from an overflow, if clicked beyond column (or row) 223. E.g. column 227 is converted to byte 3, which presumably (I haven't verified) will be interpreted as ^C interrupt by the line discipline (see gnome-terminal counterpart issue). In such cases, no mouse event should be reported.
There are 4 bits under TIOCL_SELBUTTONMASK that denote two modifiers, and the mouse button that was pressed or a generic value for releasing.
There are 3 keyboard modifiers that should be of interest, xterm and other emulators support denoting Ctrl, Shift and Alt. Now, gpm might decide that it doesn't support all three, maybe forces its own selection operation and does not forward anything to the apps whenever a modifier of its own choice is held down. However, I don't think this decision and restriction belongs to the kernel. Furthermore, which modifier this is (if at all) could be a runtime config option of gpm. I think the kernel's interface should be extended to support all three modifier keys.
The aforementioned 1006 extension adds another twist: at mouse release, the exact button that was released is reported. For this to work, the 2 bits denoting which button was pressed or a generic value for releasing any button is no longer sufficient. The interface should be extended here as well.
Mouse wheel scrolling events aren't supported. They should be.
There's a mouse extension 1002 where mouse movements are reported while a button is being held down. This is not supported by the kernel, but should.
There's a mouse extension 1003 where mouse movements are reported even when no button is being held down. I think this is somewhat debatable whether this is a good one, but I don't think it can hurt to add, and if gpm supports reporting such events via its own protocol (I don't know whether it supports) then for feature parity, in order to switch to the new design without feature regressions, this mode should be added. (Even if gpm decides not to support this mode, I think it should still be added to the kernel.)
I find it a pretty unfortunate design to piggyback on the TIOCL_SETSEL, as if mouse reporting was a special case of selection. It totally isn't. And then there are two pairs of coordinates passed, out of which only the first one is used (which is again a bit weird, with mouse dragging if anything then the second would make more sense).
In my opinion, the cleanest would be to redesign mouse reporting using a new value as the first byte (e.g. after TIOCL_GETMOUSEREPORTING the next value of 8 is free) so that it's unrelated to selection, followed by 2 shorts for the coordinates, and another for the keyboard modifiers, mouse buttons, wheel direction etc.
The ioctl_console(2) manpage says "Ioctl's are undocumented Linux internals, liable to be changed without warning." I don't know whether that's true, I'm not familiar with kernel developing practices. That being said, the old mode could still be supported for a couple of years along with the new one.
Once done, gpm could catch up and start using these revised ioctl parameters, thus hopefully arrive to feature parity with its current features and today's popular graphical terminal emulators.
Roadmap
I hope that if there's a proof-of-concept gpm implementation using the new kernel features, kernel developers would probably quickly approve the kernel patch that expands the interface, fixes the overflow in 1000 mode, adds support for the 1006 format (including reporting the released button), adds support for the 1002 and 1003 tracking extensions, adds support for wheel scroll events and the missing modifier.
Stable kernels are released quite frequently, about every 2 months. Distributions also tend to ship pretty new kernels. gpm wouldn't have to wait long until it could start shipping the new design.
In order for distributions to pick up the newest version, it would probably need to release a new 1.20 and a new 1.99.x too. Or, even better: I think that after this architectural change it'll deserve to be called 2.0.
(I would even go as far as considering to deprecate and eventually remove the current separate library, header files, public interface. Not sure if gpm-root would still require it. Regular apps won't and shouldn't.)
Once distributions start shipping gpm 2.0, apps and the ncurses library could begin to clean up their source and remove direct gpm dependency. Some apps that have a hardwired list of terminals supporting mouse would need to add linux to that list. As stated in the aforementioned 18 year old patch, the linux terminfo entry could also add an entry for mouse support. Until these apps are updated, they would of course happily use the old interface, there wouldn't be any outage.
Outro
Now, we'd only need someone enthusiastic enough to push this through. :-)
I'm really wondering, how come basically nothing changed in the last 18 years? Is the current state considered "good enough" by most users? Is it that the console is either used as a fallback, should the graphical system be inaccessible, e.g. during a disaster recovery; or it's mainly used by power users; in either of these two cases mouse support isn't important?
Did I miss anything? Is there a fundamental problem with the approach I'm endorsing? Would it have downsides, regressions in some aspects that I don't foresee?
I hope I could help by giving an overview of the current state and how I think it should ideally look like, and maybe somebody will take care of this. (Maybe it's the complexity and hence the challenge that'll inspire someone to do this!) Or maybe this issue will stay here unresolved for another 18+ years... We'll see. :-)
Thanks for the detailed post.
Unfortunately the answer is there are not many people are interested in console mouse support.
Your ideas are interesting, but currently I doubt anyone will put a lot of energy into this.
That said, prove me wrong, if you want to.
Thanks for your response!
I myself used the console + gpm a lot about 15-20 years ago, but then I switched to graphical terminal emulators. One part of the story is that they are in most aspects far superior to the console, another part is that on the graphical system one can do other tasks as well (such as running a graphical web browser), and even copy-paste across the terminal emulator and those other software. (A third, unfortunate aspect is that switching between the console and X is a terrible experience.) I personally look at the console as a last resort in case things go really wrong.
I'm not sure about other people, but what you say seems to align with my guess.
So, let my thoughts sit here until someone feels like addressing these. If no one cares enough to address them then probably it wouldn't be engineering time wisely spent, that's also fine by me.