SSH Error - handshake failed

Question

SSH Error - handshake failed

kscarlett opened this issue 5 years ago · 37 comments

I tried adding a server today with the following command and the resulting output:

$ k3sup install --context k3s-dev --ip 163.172.147.187 --user kscarlett --ssh-key ~/.ssh/id_rsa
Public IP: <ip>
ssh -i /Users/kscarlett/.ssh/id_rsa kscarlett@<ip>
Error: unable to connect to <ip>:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

The strange thing is that when I copy-paste the SSH command it prints, it logs me in just fine. Of note is that when I SSH into the server, it takes ~5 seconds, while k3sup fails immediately.

Expected Behaviour

Successful SSH authentication, just as I get manually.

Current Behaviour

Near-immediate failure of the SSH command.

Possible Solution

Steps to Reproduce (for bugs)

Seems like normal workflow - environment issue?

Context

I am unable to create a new server.

Your Environment

Local

OS: macOS 10.14.6
SSH: OpenSSH_7.9p1, LibreSSL 2.7.3

Server

OS: Ubuntu 18.04.3 LTS
SSH: OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
Hosted at Scaleway (C2L)

Answer 1 · 2019-10-23T11:02:16.000Z

It seems like it was fixed by removing the user and letting it use root instead. This puzzles me as it worked correctly with the user for me.

Answer 2 · 2019-10-27T20:06:41.000Z

Glad you found a way around it. I wonder why it took 5 seconds to log in via SSH?

Answer 3 · 2019-11-20T11:06:22.000Z

This is an error in pkg/cmd/join.go. Around Line 81 it says:

		config := &ssh.ClientConfig{
			User: user,
			Auth: []ssh.AuthMethod{
				authMethod,
			},
			HostKeyCallback: ssh.InsecureIgnoreHostKey(),
		}

The variable user should be serverUser:

		config := &ssh.ClientConfig{
			User: serverUser,
			Auth: []ssh.AuthMethod{
				authMethod,
			},
			HostKeyCallback: ssh.InsecureIgnoreHostKey(),
		}

Answer 4 · 2019-11-30T23:01:21.000Z

Feel free to send a PR?

Answer 5 · 2020-08-16T17:16:51.000Z

@cbrachem would you like to send a PR?

Answer 6 · 2020-08-16T17:16:58.000Z

/add label: support,good first issue,help wanted

Answer 7 · 2020-08-30T07:36:47.000Z

I am having the same issue on OSX

Answer 8 · 2020-09-08T21:48:51.000Z

happy to submit a PR for this @alexellis if it's still needed?

Answer 9 · 2020-09-18T01:13:01.000Z

I'm also getting this error, running on Ubuntu 20.04 on an RPi4

Answer 10 · 2020-09-23T16:58:21.000Z

@alexellis

@cbrachem would you like to send a PR?

I can't, you fixed the issue I was having last December with commit ecd37d0. What's there now must be another thing.

Answer 11 · 2021-05-14T08:27:52.000Z

Was facing the same issue on Ubuntu 20.04 on an RPi4

specifying the private key path worked (make sure public key is in authorized_keys)

k3sup install --ip $IP --user ubuntu --ssh-key ~/.ssh/id_rsa

Answer 12 · 2021-06-17T08:33:07.000Z

I have the same error, on Debian 10. It almos looks like the --user is ignored.

$ k3sup version
...
Version: 0.11.0
Git Commit: fd9dfeaa6cd32f0d048f33705a04c14ca4aa3550

SSH connections work:

root@mediaserver ~ $ ssh admin@192.168.80.114  'ls'
total 16
drwx------. 3 admin admin  95 Jun 17 08:18 .
drwxr-xr-x. 4 root  root   31 Jun 17 07:54 ..
-rw-------. 1 admin admin  36 Jun 17 08:18 .bash_history
-rw-r--r--. 1 admin admin  18 Jun 14 14:52 .bash_logout
-rw-r--r--. 1 admin admin 141 Jun 14 14:52 .bash_profile
-rw-r--r--. 1 admin admin 492 Jun 14 14:52 .bashrc
drwx------. 3 admin admin  31 Jun 17 07:54 .ssh

Connecting with k3sup doesn't:

$ k3sup install  --ip 192.168.80.114
Running: k3sup install
2021/06/17 10:31:17 192.168.80.114
Public IP: 192.168.80.114
Error: unable to connect to 192.168.80.114:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Answer 13 · 2021-07-22T14:12:42.000Z

I am using two Fedora 33 Vm's on virtualbox.. I also have this issue.
My VM's are 192.168.56.106 and 192.168.56.107. I've setup so that I can ssh from one to the other as user dsargrad. I am not prompted for a password because authorized_keys is setup properly.

Answer 14 · 2021-07-22T14:30:20.000Z

I can't even seem to authenticate as root.. Though I've also added root to authorized_keys.

Guidance?

Answer 15 · 2021-08-09T05:19:27.000Z

I can't even seem to authenticate as root.. Though I've also added root to authorized_keys.

Guidance?

Same problem. No idea. Running Raspbian.

Answer 16 · 2021-09-16T17:09:24.000Z

Same problem here on MacOS, using --user --ssh-key and --sudo to connect to a Fedora CoreOS.

Answer 17 · 2021-09-24T01:08:56.000Z

I was following this youtube tutorial https://www.youtube.com/watch?v=qsy1Gwa-J5o, same problem for me running on MacOS Big Sur

192.168.1.112 is a rpi running ubuntu

k3sup install --ip 192.168.1.112 --user ubuntu --ssh-key ~/.ssh/id_rsa                                     
Running: k3sup install
2021/09/23 21:05:06 192.168.1.112
Public IP: 192.168.1.112
Error: unable to connect to 192.168.1.112:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Answer 18 · 2021-09-24T18:14:42.000Z

Thanks @alexellis for pointing out the instructions in readme that I somehow managed to skip over as I was scanning through it yesterday :-).

This should work

then run
go build && ./k3sup install --ip 192.168.xxx.xxx --user ubuntu

Answer 19 · 2021-09-24T18:18:13.000Z

There should be no need to build k3sup from source to use it.

ssh-copy-id is in the documentation (Readme) as a required step.

Answer 20 · 2021-09-24T18:32:35.000Z

Right, I did try it without builld.

Answer 21 · 2021-09-29T04:00:32.000Z

I have an almost similar issue with @playground when I want to apply k3sup join command directly from Server(RPi) terminal.

I have RPi running as a Server (master) node with multiple IP :

192.168.XXX.XXX to connect to internet
172.19.XXX.XXX to connect to Slaves node.

I used k3sup install --local to install directly on my Server and I worked.

If I used k3sup install --ip 192.168.XXX.XXX --user pi --ssh-key ~/.ssh/id_rsa command or k3sup install --ip 172.19.XXX.XXX .. like @playground both failed as below:

Error: unable to connect to 192.168.XXX.XXX:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Error: unable to connect to 172.19.XXX.XXX:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

I am using NAT mode to connect Server and Slaves, I've already done ssh-copy-id to the Slave and I don't have a problem to ssh from Server manually.

But when I want to join the Slave nodes using k3sup join --ip 172.19.XXX.ZZZ --server-ip 172.19.XXX.XXX --user pi command or k3sup join --ip 172.19.XXX.ZZZ --server-ip 192.168.XXX.XXX --user pi, it were failed

Error: unable to connect to (server) 172.19.XXX.XXX:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Error: unable to connect to (server) 192.168.XXX.XXX:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

I already followed the readme documentation and the blog but still did not work for me.

Do you have any guidance ?

Answer 22 · 2021-10-02T11:50:16.000Z

i think i had similar problems connecting to newer fedora hosts.
login using the ssh binary worked with my rsa key but not using k3sup.
I asume the used go crypto library just uses a hash algorythm for signing that the default sshd config doesnt allow anymore like sha1.
Using a ecdsa keypair worked fine for me as a workaround.

Answer 23 · 2021-10-02T12:16:09.000Z

@kareotoko @mgolisch have you run ssh-copy-id first?

Answer 24 · 2021-10-04T15:26:59.000Z

sure, otehrwise login using the ssh binary would not have worked either.
In my case it seems to boil down to the fedora sshd config not allowing rsa publickey auth with sha1.
the ssh binary correctly negotiates the authentication/signing method with the server and chooses sha2-256 with that rsa key.
but k3sup seems to use sha1 and this is not allowed by the sshd config on newer fedora hosts.

here is an excerpt from /var/log/secure.log on the server:
Oct 4 16:49:39 fedora sshd[1036]: Accepted publickey for root from 192.168.122.1 port 47510 ssh2: RSA SHA256:CRWfVpwTWx1J0Dmh5YCPvtPFFxUl3vxrZeo0tbp/bZ8
Oct 4 16:49:39 fedora sshd[1036]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0)
Oct 4 16:53:33 fedora sshd[1039]: Received disconnect from 192.168.122.1 port 47510:11: disconnected by user
Oct 4 16:53:33 fedora sshd[1039]: Disconnected from user root 192.168.122.1 port 47510
Oct 4 16:53:33 fedora sshd[1036]: pam_unix(sshd:session): session closed for user root
Oct 4 16:53:33 fedora sshd[2314]: userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]
Oct 4 16:53:33 fedora sshd[2314]: Connection closed by authenticating user root 192.168.122.1 port 47512 [preauth]

it shows the ssh binaries login which uses rsa with sha2-256 and then the denied login atemp from k3sup.

It seems to be an underlying issue in the used golang crypto library that doesnt negotiate the signing algorythm with the server but instead allways uses sha1 for rsa keys.
golang/go#39885

As already said the easy workaround is to not use an rsa keypair

Answer 25 · 2021-10-07T17:03:35.000Z

Hi guys, I just found k3sup join has --user for agent username and --server-user for server username. The --server-user is missing in the tutorial.
k3sup join --ip $AGENT_IP --user $AGENT_USER --server-ip $SERVER_IP --server-user $SERVER_USER

Answer 26 · 2021-11-02T23:51:08.000Z

Hi guys, I just found k3sup join has --user for agent username and --server-user for server username. The --server-user is missing in the tutorial. k3sup join --ip $AGENT_IP --user $AGENT_USER --server-ip $SERVER_IP --server-user $SERVER_USER

This is it, this is what actually worked for me

Answer 27 · 2021-11-30T17:24:34.000Z

I have added some lines to the readme that might resolve some common issues: point users to the --server-user parameter and adresses the current problem with RSA keys in PR #359.

Answer 28 · 2021-12-09T13:30:39.000Z

Closing since #359 was merged.

Answer 29 · 2022-02-06T13:42:04.000Z

This issue is still present in k3sup install;
What I did:

Created the rsa with ssh-keygen
Copied it using ssh-copy-id

Ssh login works normal, but in k3sup it doesn't, the same as the OP did report

Answer 30 · 2022-02-11T22:35:07.000Z

This issue is still present in k3sup install; What I did:

Created the rsa with ssh-keygen

Copied it using ssh-copy-id

Ssh login works normal, but in k3sup it doesn't, the same as the OP did report

Have you ever tried: ssh-copy-id user@IP

Answer 31 · 2022-03-09T13:36:06.000Z

In addition to the key being RSA I had another issue: I was tunneling into the machine (let's call it a bastion) where I was running k3sup and I had agent forwarding on so when I ran ssh-copy-id it copied my bastion key and not the key from the machine I was in, but k3sup was using the key from the bastion and it failed. Took a while to figure it out...

Answer 32 · 2022-03-24T14:08:08.000Z

As mentioned by @JGabrielGruber this issue is still present, I did the same things, keys generation and ssh-copy-id.
I've tried on Manjaro and Alpine as clients.
On the Server side (CentOS Stream 9) I have this:

userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]

SSH login using OpenSSH client with the same key works without problems.

Answer 33 · 2022-04-28T12:04:13.000Z

Can confirm, still present with ssh-copy-id, ssh ubuntu@192.168.1.30 works, but k3sup install does throws

k3sup version
Version: 0.11.3

k3sup install --ip $IP --ssh-key "~/.ssh/id_rsa" --user "ubuntu"
Running: k3sup install
2022/04/28 07:55:39 192.168.1.30
Public IP: 192.168.1.30
Error: unable to connect to 192.168.1.30:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Edit:

k3sup install --ip $IP --user ubuntu
Running: k3sup install
2022/04/28 08:07:22 192.168.1.30
Public IP: 192.168.1.30
Error: unable to connect to 192.168.1.30:22 over ssh: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Answer 34 · 2022-04-28T12:06:21.000Z

Hi @dreallday I cannot reproduce this and used the very same above recently on my own Raspberry Pis to install K3s using K3sup.

--ssh-key "~/.ssh/id_rsa" won't be parsed like you think and should be: --ssh-key "$HOME/.ssh/id_rsa", however this doesn't need to be set as it's a default.

Feel free to try that out, then check the permissions of your .ssh folder on both ends, and the permissions of the files too, this could also prevent the above from working, but as I say, it works as expected for me.

Alex

Answer 35 · 2022-04-28T12:13:34.000Z

any variation of the k3sup throws the same ssh error

.ssh directory: 700 (drwx------)
public key (.pub file): 644 (-rw-r--r--)
private key (id_rsa): 600 (-rw-------)

this is a fresh install on ubuntu 20.04 with leader node running 22.04

Answer 36 · 2022-04-28T12:15:51.000Z

I cannot reproduce this, so I don't know how to help you at this point.

Can you open your own issue, because there's so much information lacking here? Perhaps we'll learn something from that.

Alex

Answer 37 · 2022-04-28T12:16:24.000Z

/lock: Users should create their own issues for support. My GitHub Sponsors get priority.