Nested tmux session over ssh

Question

Nested tmux session over ssh

sourya7 opened this issue 5 years ago · 21 comments

Hi, first of all, this is pretty sweet. One question I have is about nesting over ssh. In my client, I have tmux and I want to be able to ssh over to a server, run tmux and switch between the two. Is this something that is possible? I'm not very competent with tmux to know if we can even achieve this but it would be pretty cool if we can. If not, I can live with two terminals.

Answer 1 · 2020-01-28T20:59:24.000Z

I've thought about that, it should possible to notify an inner tmux session over SSH by sending a key (we send Alt + F12). However notifying the outer tmux session is more of an issue since we do this by writing on the tmux socket specified through an environment variable (TMUX_PARENT). However, that socket is on another machine.
We should redesign this approach and try to switch to a key-based approach to communicate to outer tmux too. However, this is not obvious.

Answer 2 · 2021-02-08T07:42:46.000Z

Hi there.
I'll preface this suggestion with the following acknowledgement- What I'm proposing adds a really decent amount of complexity, and turns this from being a tmux conf into being something more complex.

One way to potentially support this is to use remote port fort forwarding in the ssh session, as in:

# Tunnel client's localhost:8008 => remote-host's localhost:8008
ssh -R 8008:localhost:8008 remote-host

OR more robustly in the ssh config

Host remote-host
   RemoteForward 8008 localhost:8008

On the remote host, we can redirect the TMUX_PARENT socket to localhost:8008, probably using socat or something.

On the client (local) machine, we can use socat to redirect localhost:8008 to our actual TMUX socket.

There's a few caveats - A lot of enterprise/corporate servers are going to restrict what you can & can't do with ssh port forwarding (legitimate security reasons)- there's a decent chance you just can't for some servers.

To enable portforwarding, the remote's sshd_config needs to have:

AllowTcpForwarding yes

To allow remote forwarding, it needs

GatewayPorts yes

Answer 3 · 2021-02-08T07:51:48.000Z

Actually, according to
https://serverfault.com/a/753733/615908 since OpenSSH 6.7 , you can forward local unix sockets directly (this blew my mind).

Of course, a lot of enterprise servers run old versions of OpenSSH (My workplace's HPC runs OpenSSH 5.3), so we'll have to settle with using socat to redirect in those instances.

Since ssh_configs don't allow for env substitution (for example, substituting TMUX as the socket to forward), chances are, you're going to need either an alias, or a script/function to service this connection, so that you can forward the correct socket dynamically.

EDIT

~~actually looks like you can only localforward unix domain sockets, so without some more redirection via socat, chances are you can't directly do this - checking now~~

nevermind, apparently it's valid to remote forward unix sockets. SO cool my gosh.

Answer 4 · 2021-02-08T08:02:13.000Z

Since ssh_configs don't allow for env substitution (for example, substituting TMUX as the socket to forward), chances are, you're going to need either an alias, or a script/function to service this connection, so that you can forward the correct socket dynamically.

Answer 5 · 2021-02-08T08:23:06.000Z

I didn't know about forwarding UNIX sockets! Nice!
Before thinking about embedding this in ssh_config or supporting old SSH versions, I'd like to see this work with an ssh wrapper we can provide (e.g., ssh-nested-tmux).

Have you tried it? Does it work?

Answer 6 · 2021-02-08T08:25:14.000Z

Here's a (probably) working solution for both client & server OpenSSH >= 6.7:

#!/usr/bin/env bash

# -R remote_socket:local_socket
ssh -R "/tmp/tmux_parent:${TMUX}"

on the remote host, do the following:

# append this env var set to the ssh rc
cat << EOF >> ~/.ssh/rc
# check socket exists
if [ -S /tmp/tmux_parent ]; then 
   echo "export TMUX=/tmp/tmux_parent" >> ~/.ssh/rc
fi
EOF

I think this should do the trick

EDIT Just saw your reply, I'd finished writing this wrapper/conf before I saw it

@aleclearmind

I'd like to see this work with an ssh wrapper we can provide (e.g., ssh-nested-tmux).

Have you tried it? Does it work?

Well, here it is, but I haven't tried it yet, but also, there's a few details that you probably understand better than me about what env vars influence what.

I'd like your feedback on whether we should be setting:
TMUX vs TMUX_PARENT to /tmp/tmux_parent in our ~/.ssh/rc

For context on ~/.ssh/rc , from the man page:

~/.ssh/rc
Commands in this file are executed by ssh when the user logs in, just before the user's shell (or command) is started. See the sshd(8) manual page for more information

(this executes at ssh login, right before the shell starts)

Answer 7 · 2021-02-08T08:33:38.000Z

Mmmh, why do you rename the remote socket? This does not support multiple nested sessions over the same machine.
Also, can't you use SendEnv instead of ~/.ssh/rc?

Answer 8 · 2021-02-08T08:37:46.000Z

Mmmh, why do you rename the remote socket? This does not support multiple nested sessions over the same machine.

Good point, renamed because I'm not sure what we can write to, but I guess we can just forward the socket 1:1, meaning we can directly forward TMUX as well

Also, can't you use SendEnv instead of ~/.ssh/rc?

I think so, I'm not sure about portability (requires sshd_config to AcceptEnv yes), but assuming you have root on both machines, that's simpler, and would work better for dynamically forwarding TMUX if we don't rename the socket.

I'll rewrite the script & hide the previous attempt

Answer 9 · 2021-02-08T08:41:53.000Z

~~Local ~/.ssh/config~~

Redundant

Host remote-host
   SendEnv TMUX
   # Maybe wildcard? TMUX_*

#!/usr/bin/env bash

# -R remote_socket:local_socket
ssh -R "${TMUX}:${TMUX}" -o SendEnv=TMUX  remote-host

Answer 10 · 2021-02-08T08:42:43.000Z

-o SendEnv=TMUX?

Answer 11 · 2021-02-08T08:45:08.000Z

-o SendEnv=TMUX?

Updated

Answer 12 · 2021-02-08T08:46:45.000Z

I actually haven't applied nested-tmux to my own tmux conf, btw, so I can't test this until I do so, if you're able to test it out I'd love to hear about it- Thank you for your feedback!

If it works, I'd love to be the one to open the PR if that's okay

Answer 13 · 2021-02-08T08:50:31.000Z

Regarding directly forwarding the tmux socket, I just checked and it looks like my UID is encoded into the directory of the tmux socket, we may need to hack around that a bit for systems where the client uid != remote uid, idk

Answer 14 · 2021-02-08T09:13:20.000Z

Please, reduce the amount of messages, test things and then make a PR.
Otherwise I'll look into that when I get the chance.

Answer 15 · 2021-04-07T03:36:08.000Z

Are there possibly any other ways to do this? I don't know about tmux's security model, but forwarding sockets leaves me a bit uncomfortable. - or is that unwarranted?

Answer 16 · 2021-04-08T11:28:22.000Z

Yeah, it'd be nice to limit what can be communicated back to the client only a signal to reactivate the outer layer.

Answer 17 · 2021-04-08T14:48:56.000Z

Ok so there are (at least?) two ways to do this

wait for a "signal" (side-channel) from the inner session to change the tmux state
directly modify the parent state
use terminal escape sequences?

nested-tmux curently does the latter, which is what introduces the socket requirement.
The problem with the former is you need to find a signal, that you can observe, which is obscure enough as to generally not impair functionality if it is co-opted. This also limity flexibility and youhave to define a protocol to use over the sidechannel.

Looking through the tmux man page,

All the notifications listed in the CONTROL MODE section are hooks (without any arguments), except %exit.  The following additional hooks are available:

Items that caught my eye for tunneling information: pane-set-clipboard,
Things that are 1-bit signals seem tedious, but one-hot encoding two 1-bit signals should be feasible at least for triggering "up/down" navigation.

If the clipboard hook combined with OSC 52 (apparently some sort of clipboard copy escape sequence; needs more research; example: https://gist.github.com/yudai/95b20e3da66df1b066531997f982b57b ) allows us to be notified on data transfer via the clipboard, we could parse arbitrary information. The main downside i see is having to deal with clobbering the clipboard, but apparently the xterm escape sequence (section P c ; P d section in https://www.xfree86.org/current/ctlseqs.html) can specify a target buffer? I'm not sure if that's the same thing though. tmux/tmux#1477 also looks very relevant to this approach.

Anyway lot of this seems rather hacky and maybe it's worth asking the tmux developer for suggestions once there's a clear question.

I don't immediately see any other hooks we could use.

Another question is how to handle something with the ssh connection breaking, timing out, or accidentally outputting bad data on the terminal. This is generally a risk with in-band signaling.

Answer 18 · 2021-04-08T15:01:59.000Z

Making this work also would/could allow assimilating existing (nested) tmux sessions?
I've done some experiments trying to do that but I had some issues I don't understand yet.

Answer 19 · 2021-04-08T21:51:10.000Z

Should we just try to get upstream to support nested tmux better by design? :P

Answer 20 · 2021-04-12T05:36:32.000Z

Should we just try to get upstream to support nested tmux better by design? :P

By upstream, do you mean nicm & the tmux maintainers?

They're great maintainers, but they're pretty hard to influence, I guess you can try your luck lmao, but I'm sure this has already been raised with them