key exhange negotiation failed though client and server share some protocols

Gitea Version

1.15.3

Git Version

2.33.0

Operating System

CentOS 7

How are you running Gitea?

After a openssh client upgrade, push to gitea

Database

SQLite

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

using built-in SSH server with below config:

[server]
SSH_SERVER_CIPHERS      = aes128-ctr aes192-ctr aes256-ctr aes128-gcm@openssh.com arcfour256 arcfour128
SSH_SERVER_KEY_EXCHANGES = curve25519-sha256 diffie-hellman-group1-sha1 diffie-hellman-group14-sha1 ecdh-sha2-nistp256 ecdh-sha2-nistp384 ecdh-sha2-nistp521 curve25519-sha256@libssh.org ssh-rsa

both client and server have common algorithm, but report no common algorithm for key exchange

2021/09/29 10:46:19 modules/ssh/ssh.go:259:sshConnectionFailed() [W] Failed connection from 10.118.13.125:50358 with error: ssh: no common algorithm for key exchange; 
client offered: [curve25519-sha256 curve25519-sha256@libssh.org ecdh-sha2-nistp256 ecdh-sha2-nistp384 ecdh-sha2-nistp521 diffie-hellman-group-exchange-sha256 diffie-hellman-group16-sha512 diffie-hellman-group18-sha512 diffie-hellman-group14-sha256 ext-info-c], 
server offered: [curve25519-sha256 diffie-hellman-group1-sha1 diffie-hellman-group14-sha1 ecdh-sha2-nistp256 ecdh-sha2-nistp384 ecdh-sha2-nistp521 curve25519-sha256@libssh.org]

Screenshots

No response

@rockmenjack 1.5.3 is quite old - we can not hunt down the issue on this version - its ancient

please upgrade to latest stable 1.15.3

@rockmenjack 1.5.3 is quite old - we can not hunt down the issue on this version - its ancient

please upgrade to latest stable 1.15.3

Oh, that was a typo, I was supposed to mean 1.15.3

can reproduce on main ...

I'm not sure if it's related but after the upgrade I started getting:

Unable to negotiate with XYZ port 22: no matching host key type found. Their offer: ssh-rsa
fatal: Could not read from remote repository.

I'm also using built-in SSH server and I didn't configure neither SSH_SERVER_KEY_EXCHANGES nor SSH_SERVER_CIPHERS previously, so I suppose it uses defaults.

I'm having the same issue as @synaptiko above. Adding HostkeyAlgorithms ssh-rsa to my ~/.ssh/config for the host fixed the host key type problem, but my keys don't work. Seems like the internal ssh server just isn't working as it used to.

EDIT: Turns out I'm running gitea 1.14.5, so nevermind, I should probably be running the freshest code before complaining 😄 Worked around it personally by just switching to the system's ssh server instead of gitea's internal one.

Looks like the same issue as here

gogs/gogs#6623

Probably golang/go#37278 and it happens with openssh >= 8.8

From https://www.openssh.com/releasenotes.html :

by default. This change has been made as the SHA-1 hash algorithm is
cryptographically broken, and it is possible to create chosen-prefix
hash collisions for <USD$50K [1]

Workaround is to specify SSH parameter in ~/.ssh/config:

Host mygiteahost.com
	Port 2222
	HostKeyAlgorithms +ssh-rsa
	PubkeyAcceptedAlgorithms +ssh-rsa
	User gitea

Apparently ssh-rsa host key algorithm (and apparently also the signature algorithm) was declared deprecation a year ago and it finally has an effect.

Also to add insult to injury, it seems like gitea's internal ssh server is hard-coded to only generate ssh-rsa keys so I can't easily force it into rsa-sha2-* or other key types/algorithms, and the easiest workaround seems to be enabling ssh-rsa back as @nougad suggested.

Found some informations here : https://dev.to/cloudx/why-openssh-8-8-cannot-find-a-host-key-type-if-ssh-rsa-is-provided-49i

Please could you tell us whether you are running the docker, docker-rootless or are running a gitea binary. Please confirm if you are using the internal SSH e.g. (START_SSH_SERVER=true) or an external SSH (START_SSH_SERVER=false)

Oh damn - this is going to be annoying and difficult to fix.

Fundamentally there is a problem in the way that Go's x/crypto/ssh library does the handshake.

Please could you tell us whether you are running the docker, docker-rootless or are running a gitea binary. Please confirm if you are using the internal SSH e.g. (START_SSH_SERVER=true) or an external SSH (START_SSH_SERVER=false)

@zeripath in my case we are running the gitea binary on last debian version with the internal SSH server (managed by gitea).

@Poyoman39 the attached PR will solve the issue.

For people looking for a workaround way to fix this on their server without people having to make changes to ~/.ssh/config on the client end:
If you manually generate and ED25519 keypair ssh-keygen -t ed25519 and reference them with the SSH_SERVER_HOST_KEYS setting in app.ini it will force a better signature algorithm based on said key type.

IMO, it's worth looking at moving to ED25519 or another elliptic curve algorithm as the default for Gitea, as RSA is getting closer to being totally broken all the time.

IMO, it's worth looking at moving to ED25519 or another elliptic curve algorithm as the default for Gitea, as RSA is getting closer to being totally broken all the time.

and probably have a configuration option/GUI option on the admin interface so ssh-keygen will not be required to generate arbitrary key type.

My company's entire ed25519 keys stopped working lmao
Why is everything set to RSA? it's not 2010...

>SSH_SERVER_HOST_KEYS: ssh/gitea.rsa, ssh/gogs.rsa: For the built-in SSH server, choose the keypairs to offer as the host key. The private key should be at SSH_SERVER_HOST_KEY and the public SSH_SERVER_HOST_KEY.pub. Relative paths are made absolute relative to the APP_DATA_PATH. If no key exists a 4096 bit RSA key will be created for you.

~~uhh what, anyone able to tell me what to do to just get ed25519 keys going? idc about any other ones.~~

nvm all you need to do is generate a new ed25519 key and set it with SSH_SERVER_HOST_KEYS as the guy above said.. can't believe you guys are using RSA key lmao....

also running the via the binary (not docker) @zeripath but seems you figured it out
(using the gitea internal SSH server.. I run openSSH server already and have no issues there, as I've used ed25519 for several years.)

@dogtopus

he easiest workaround seems to be enabling ssh-rsa back as

The easiest solution is just generating an ed25519 and pointing to it, takes <10 seconds

FWIW to gitea devs
https://pkg.go.dev/crypto/ed25519

The easiest solution is just generating an ed25519 and pointing to it, takes <10 seconds

Yes but if you got a bunch of different clients connected to the server it would be a PITA to update them all to use the new public key. Although adding back RSA SHA1 on every machine is not ideal either.

Yep, ed25519 is something that should have been defaulted ~10 years ago. My entire office has to delete their known_hosts line now and accept the new key. Not really that big of a deal.

For anyone who ends up here, here is how you can easily switch over to ed25519.

Use ed25519 on the gitea side

Add SSH_SERVER_HOST_KEYS = ssh/gitea.ed25519 to your app.ini:

SSH_SERVER_HOST_KEYS     = ssh/gitea.ed25519

Don't bother setting SSH_SERVER_CIPHERS, SSH_SERVER_KEY_EXCHANGES and SSH_SERVER_MACS, that is not necessary. However, in case you find it necessary to do so, please consult man sshd_config and ssh -Q cipher|key|key-sig.

In case the keys don't get generated automatically, move to the ssh directory in your gitea data dir and generate the keys manually:

cd /var/gitea/ssh             # Change to your needs
ssh-keygen -t ed25519 -f gitea.ed25519

Then restart gitea, and look for the following entry in the log

2021/10/19 07:11:14 modules/ssh/ssh.go:312:Listen() [I] Adding SSH host key: /data/gitea/ssh/gitea.ed25519

Perhaps the gitea config cheat sheet is useful, if the above stated suggestions don't work for you and you need to dive deeper into the configuration options.

Force ssh to use ed25519 for your gitea server

In my case the above stated steps were enough to fix the issue. If for some reason, you ssh needs to be forced to use ed25519, then see the following template for your ~/.ssh/config file

HOST gitea.local gitea
    HOSTNAME gitea.local
    PreferredAuthentications publickey
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Update

Addendum: Friend of mine reported, gitea will generate a rsa key by default, so it looks like it's mandatory to generate the keys manually