When using OpenSSH ProxyJump, "Allow for 3 hours" only working for jump host

Question

When using OpenSSH ProxyJump, "Allow for 3 hours" only working for jump host

ldcasillas-progreso opened this issue 8 years ago · 9 comments

ldcasillas-progreso commented 8 years ago

I'm using the iOS app version 2.1.1, and these for the other apps:

$ kr --version
kr version 2.1.2

$ ssh -V
OpenSSH_7.3p1, LibreSSL 2.4.1

The issue I'm seeing is when I try to connect to an SSH destination through an intermediate "jump host," using a configuration similar to this (edited for confidentiality and to remove (hopefully) irrelevant details):

Host jump-host
    HostName jumphost.companyname.local

Host destination-host
    Hostname 10.42.0.30
    ProxyJump jump-host

Host *
    PKCS11Provider /usr/local/lib/kr-pkcs11.so
    ProxyCommand /usr/local/bin/krssh %h %p
    IdentityFile ~/.ssh/id_kryptonite

My Kryptonite public key is installed in the authorized_hosts file of both the jump host and the destination host, and I can successfully connect to destination-host with the command ssh destination-host. The problem is that after a recent version upgrade, the "Allow for 3 hours" option in the iOS app only works for the jump host. Here's the sequence of events for a "cold" connection:

I use the command ssh destination-host from a computer paired with the Kryptonite iOS app
The connection to jump-host is initiated, and I'm alerted on my phone to approve of an authentication request from jump-host.companyname.local.
I choose "Allow for 3 hours."
The connection to the jump host is established.
A connection to the destination host is initiated through a jump host tunnel, and and I'm alerted on my phone to approve of an authentication request from unknown host.
I choose "Allow for 3 hours."
The connection to the destination host is established.

But when I reattempt the connection immediately thereafter:

ssh destination-host
The connection to jump-host is initiated, I'm alerted on my phone that an authentication request from jump-host.companyname.local was automatically approved, and the connection to the jump host is established.
A connection to the destination host is initiated through a jump host tunnel, and and I'm alerted on my phone to approve of an authentication request from unknown host.
I'm once more presented with the phone prompt to allow or reject the authentication request.

Additional observations:

My iOS app's known hosts shows the jump host (jump-host.companyname.local), but not the destination host.
When I run a verbose SSH command, I can see the jump host identifying itself by FQDN (jump-host.companyname.local), but the destination host by IP address.
The destination host's IP address is in an IPv4 private address range.

Some possibly relevant debug messages from the ssh client (from a "warm" connection attempt, and edited for confidentiality):

debug1: Executing proxy command: exec /usr/local/bin/krssh jump-host.companyname.local 22

debug1: Authenticating to jump-host.companyname.local:22 as 'username'

debug1: Host 'jump-host.companyname.local' is known and matches the RSA host key.
debug1: Found key in /Users/username/.ssh/known_hosts:236

debug1: Offering RSA public key: /Users/username/.ssh/id_kryptonite
debug1: Server accepts key: pkalg ssh-rsa blen 535
Kryptonite ▶ Requesting SSH authentication from phone
Kryptonite ▶ Success. Request Allowed ✔
debug1: Authentication succeeded (publickey).
Authenticated to jump-host.companyname.local (via proxy).
debug1: channel_connect_stdio_fwd 10.20.0.75:22
debug1: channel 0: new [stdio-forward]

debug1: Authenticating to 10.42.0.30:22 as 'username'

debug1: Offering RSA public key: /Users/username/.ssh/id_kryptonite
debug1: Server accepts key: pkalg ssh-rsa blen 535
Kryptonite ▶ Requesting SSH authentication from phone
Kryptonite ▶ Phone approval required. Respond using the Kryptonite app
Kryptonite ▶ Success. Request Allowed ✔
debug1: Authentication succeeded (publickey).
Authenticated to 10.42.0.30 (via proxy).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.

Answer 1 · 2017-06-08T23:52:49.000Z

Reading some of the other issues I managed to find the toggle to disable forced alert on unknown hosts (#52). Turning this off made authentication succeed.

My first remark is boy, that toggle sure was well hidden—you have an UX issue here for sure.

The second remark is that the behavior seems to be wrong; this is not really an unknown host as I understand it, it's a host that has already presented its public key to us repeatedly.

Answer 2 · 2017-06-12T20:26:53.000Z

Hi Luis, when using ProxyJump, krssh is only invoked for the first connection to the jump host and not the destination.

For this reason, we added an optional proxy argument to krssh that has the same effect as ProxyJump, but also parses the incoming signature from the destination host.

Instead of ProxyJump jump-host, try changing this line to ProxyCommand krssh -p "ssh -W %h:%p jump-host" -h %h

We realize this is not well documented and will be releasing a blog post explaining this soon. Let us know if this helps you.

Answer 3 · 2017-06-12T21:41:48.000Z

I tried ProxyCommand krssh -p "ssh -W %h:%p jump-host" -h %h as you suggested. First observation: in my example jump-host is not an actual hostname, but an alias defined in the SSH client config file of the host where I run krssh. So the command gives me this:

$ ssh -v destination-host
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 16: Applying options for destination-host
debug1: /Users/username/.ssh/config line 123: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: /etc/ssh/ssh_config line 102: Applying options for *
debug1: Executing proxy command: exec krssh -p "ssh -W 10.42.0.30:22 jump-host" -h 10.42.0.30
debug1: permanently_drop_suid: 1843725815
debug1: identity file /Users/username/.ssh/id_kryptonite type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/username/.ssh/id_kryptonite-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
could not connect to remote: dial tcp: lookup jump-host: no such host
ssh_exchange_identification: Connection closed by remote host

And then the session just hangs. Fix/workaround: replace jump-host in the command with the actual hostname.

Second observation: after I apply the workaround, the SSH connection into the jump host now no longer respects the "Allow for 3 hours" option. That is, when I try to connect to the destination host:

The phone app prompts me repeatedly to authenticate to "unknown host" for the jump host session, and choosing "allow for 3 hours" doesn't make it go away;
The phone app only prompts me once for authentication to the destination host, and if I choose "allow for 3 hours" it doesn't prompt me anymore.

So this suggestion that you've offered is doing something, but it's not really working. I'll close with the log for this latter case:

$ ssh -v destination-host
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 16: Applying options for destination-host
debug1: /Users/username/.ssh/config line 123: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: /etc/ssh/ssh_config line 102: Applying options for *
debug1: Executing proxy command: exec krssh -p "ssh -W 10.42.0.30:22 jump-host.companyname.local" -h 10.42.0.30
debug1: permanently_drop_suid: 1843725815
debug1: identity file /Users/username/.ssh/id_kryptonite type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/username/.ssh/id_kryptonite-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3

[Jump host login banner]

Kryptonite ▶ Requesting SSH authentication from phone
Kryptonite ▶ Requesting SSH authentication from phone
Kryptonite ▶ Phone approval required. Respond using the Kryptonite app
Kryptonite ▶ Phone approval required. Respond using the Kryptonite app
Kryptonite ▶ Success. Request Allowed ✔
Kryptonite ▶ Success. Request Allowed ✔
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Authenticating to 10.42.0.30:22 as 'username'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192) sent
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-rsa SHA256:o9I11UATo2OqP874jtF03y+Gs02fdrWkk22hb8NKlqI
debug1: Host '10.42.0.30' is known and matches the RSA host key.
debug1: Found key in /Users/username/.ssh/known_hosts:247
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received

[Destination host login banner]

debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/username/.ssh/id_kryptonite
debug1: Server accepts key: pkalg ssh-rsa blen 535
Kryptonite ▶ Requesting SSH authentication from phone
Kryptonite ▶ Success. Request Allowed ✔
debug1: Authentication succeeded (publickey).
Authenticated to 10.42.0.30 (via proxy).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: proc
debug1: Sending environment.
debug1: Sending env LC_CTYPE = en_US.UTF-8

Answer 4 · 2017-06-12T22:49:35.000Z

Thank you for gathering very detailed debugging information!

This behavior is unexpected as the tunneling ssh invocation should also read your config and replace the alias with the proper hostname. This works with a local setup I have (I can even jump through multiple hosts in a row), albeit not using a private IPv4 space but I would be surprised if that was the culprit.

Could you also change the ProxyCommand to use ssh -v so we can see the verbose output of the connection to the jump host? Similarly, it may also be informative to see the output of ssh -v jump-host.

Finally, could you please share which operating system/version you are using?

Thanks again!

Answer 5 · 2017-06-13T00:19:58.000Z

I'm using macOS Sierra 10.12.13.

$ uname -a
Darwin Luis-Macbook.local 16.4.0 Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64 x86_64

$ kr --version
kr version 2.1.2

$ ssh -V
OpenSSH_7.3p1, LibreSSL 2.4.1

Debug output with the jump-host alias:

$ ssh -v destination-host
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 16: Applying options for destination-host
debug1: /Users/username/.ssh/config line 123: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: /etc/ssh/ssh_config line 102: Applying options for *
debug1: Executing proxy command: exec krssh -p "ssh -v -W 10.42.0.30:22 jump-host" -h 10.42.0.30
debug1: permanently_drop_suid: 1843725815
debug1: identity file /Users/username/.ssh/id_kryptonite type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/username/.ssh/id_kryptonite-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 123: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: /etc/ssh/ssh_config line 102: Applying options for *
debug1: Executing proxy command: exec /usr/local/bin/krssh jump-host 22
debug1: permanently_drop_suid: 1843725815
debug1: identity file /Users/username/.ssh/id_kryptonite type 1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/username/.ssh/id_kryptonite-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
could not connect to remote: dial tcp: lookup jump-host: no such host
ssh_exchange_identification: Connection closed by remote host

[hangs until I hit ^C]

To my (not so well informed) eye, these lines jump out:

debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 123: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: /etc/ssh/ssh_config line 102: Applying options for *
debug1: Executing proxy command: exec /usr/local/bin/krssh jump-host 22

The options at line 123 of the user client config file are these, just the wildcard entry that Kryptonite appends to the file:

Host *
    PKCS11Provider /usr/local/lib/kr-pkcs11.so
    ProxyCommand /usr/local/bin/krssh %h %p
    IdentityFile ~/.ssh/id_kryptonite

Looks like the tunneling ssh invocation applies the Host * setting and its ProxyCommand, and then immediately passes the ball to krssh with the jump-host alias as its argument. Which krssh tries to resolve as a hostname:

$ krssh jump-host 22
could not connect to remote: dial tcp: lookup jump-host: no such host

Answer 6 · 2017-06-13T15:41:48.000Z

Yes it seems that SSH doesn't apply the jump-host config block to convert the alias to the correct hostname. Here are a few more debugging questions:

Do you get the same behavior when running ssh -v jump-host directly?
Which line in your ~/.ssh/config file contains the jump-host block?
What types of options are specified in /etc/ssh/ssh_config lines 20 and 102?
What shell are you using?

Answer 7 · 2017-06-14T00:00:32.000Z

Oh, boy, I had a typo. My apologies; half of what I reported is not true. I'll re-report the part that's still true now.

After I apply the workaround, the SSH connection into the jump host now no longer respects the "Allow for 3 hours" option. That is, when I try to connect to the destination host:

The phone app prompts me repeatedly to authenticate to "unknown host" for the jump host session, and choosing "allow for 3 hours" doesn't make it go away in subsequent logins;
The phone app only prompts me once for authentication to the destination host, and if I choose "allow for 3 hours" it doesn't prompt me anymore.

So the suggestion that you've offered is only shifting the user experience problem around—I'm still getting prompted in the phone app, but for the jump host instead of the destination host.

Answer 8 · 2017-06-15T04:15:49.000Z

Just to confirm with your fixed ssh config, does the Kryptonite config block come after both the jump-host and destination-host blocks in your config file?

Also a verbose output (for both the ssh invocation and the krssh ProxyCommand) for your current setup would be much appreciated :)

Answer 9 · 2017-11-01T00:35:29.000Z

Closing this for now -- feel free to open with any follow-up information or if you're still having issues