Vagrant net-ssh is assuming that any RSA key is SHA1, resulting in a connection error

Question

Vagrant net-ssh is assuming that any RSA key is SHA1, resulting in a connection error

crochat opened this issue 4 years ago · 19 comments

crochat commented 4 years ago

Vagrant version

2.2.16

Host operating system

Ubuntu 20.04

Guest operating system

Ubuntu 20.04

Debug output

D, [2021-05-01T15:09:40.735420 #92523] DEBUG -- net.ssh.authentication.session[2876340]: allowed methods: publickey
D, [2021-05-01T15:09:40.735468 #92523] DEBUG -- net.ssh.authentication.methods.none[28849a4]: none failed
D, [2021-05-01T15:09:40.735509 #92523] DEBUG -- net.ssh.authentication.session[2876340]: trying publickey
D, [2021-05-01T15:09:40.735739 #92523] DEBUG -- net.ssh.authentication.agent[289266c]: connecting to ssh-agent
E, [2021-05-01T15:09:40.735808 #92523] ERROR -- net.ssh.authentication.agent[289266c]: could not connect to ssh-agent: Agent not configured
D, [2021-05-01T15:09:40.735930 #92523] DEBUG -- net.ssh.authentication.methods.publickey[2892c20]: trying publickey (a9:7a:e7:aa:0b:89:72:91:86:81:19:4f:e0:d5:15:35)
D, [2021-05-01T15:09:40.736079 #92523] DEBUG -- net.ssh.authentication.methods.publickey[2892c20]: public key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-05-01T15:09:40.736150 #92523] DEBUG -- socket[2811e90]: using encrypt-then-mac
D, [2021-05-01T15:09:40.736196 #92523] DEBUG -- socket[2811e90]: queueing packet nr 5 type 50 len 480
D, [2021-05-01T15:09:40.736238 #92523] DEBUG -- socket[2811e90]: sent 548 bytes
D, [2021-05-01T15:09:40.739143 #92523] DEBUG -- socket[2811e90]: read 516 bytes
D, [2021-05-01T15:09:40.739245 #92523] DEBUG -- socket[2811e90]: received packet nr 5 type 60 len 448
D, [2021-05-01T15:09:40.739287 #92523] DEBUG -- net.ssh.authentication.methods.publickey[2892c20]: public key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-05-01T15:09:40.745181 #92523] DEBUG -- net.ssh.authentication.methods.publickey[2892c20]: public key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-05-01T15:09:40.745208 #92523] DEBUG -- net.ssh.authentication.methods.publickey[2892c20]: private key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-05-01T15:09:40.745260 #92523] DEBUG -- socket[2811e90]: using encrypt-then-mac
D, [2021-05-01T15:09:40.745307 #92523] DEBUG -- socket[2811e90]: queueing packet nr 6 type 50 len 896
D, [2021-05-01T15:09:40.745351 #92523] DEBUG -- socket[2811e90]: sent 964 bytes
D, [2021-05-01T15:09:40.752308 #92523] DEBUG -- socket[2811e90]: read 100 bytes
D, [2021-05-01T15:09:40.752410 #92523] DEBUG -- socket[2811e90]: received packet nr 6 type 51 len 32
D, [2021-05-01T15:09:40.752456 #92523] DEBUG -- net.ssh.authentication.session[2876340]: allowed methods: publickey
D, [2021-05-01T15:09:40.752484 #92523] DEBUG -- net.ssh.authentication.methods.publickey[2892c20]: publickey failed (a9:7a:e7:aa:0b:89:72:91:86:81:19:4f:e0:d5:15:35)
E, [2021-05-01T15:09:40.752507 #92523] ERROR -- net.ssh.authentication.session[2876340]: all authorization methods failed (tried none, publickey)

Expected behavior

It should connect without any problem, because the is not actually refused by the server.

Actual behavior

The ssh client (Vagrant, with specific patch https://github.com/hashicorp/vagrant/blob/main/lib/vagrant/patches/net-ssh.rb) is assuming that, because the ssh key is RSA, it's necessary SHA1, and it fails i.e. in "vagrant provision", despite the fact that "vagrant ssh" works well.
If you generate a ssh-key with "ssh-keygen -t rsa-sha2-512", it specifically uses the SHA2 algorithm, but this patch assumes that it can only be SHA1, because it's RSA:

"if s_ver >= Gem::Version.new("7.2") && pub_key.is_a?(OpenSSL::PKey::RSA)"

I'm working with OpenStack, but I guess it won't change in another situation. Reverting to Vagrant 2.2.15 solves (temporarily) the issue.

Answer 1 · 2021-05-01T18:57:16.000Z

@chrisroberts, @crochat
can confirm/second this issue.

ironically this breaks the case when the stock default vagrant key set is used...

simple test, with my very own ClearLinux boxes ...

Vagrant.configure("2") do |config|
  config.vm.box = "AntonioMeireles/ClearLinux"
end

as OP said, right now, reverting to previous release is the (simplest) way to go :| until a point release with this fixed is out.

Thanks in advance and a great weekend.

Answer 2 · 2021-05-06T06:50:05.000Z

Having trouble with this too. Reverted back to 2.2.14 and it works fine.

Answer 3 · 2021-05-08T14:35:48.000Z

I've run into this issue today as well with Ubuntu 21.04. Here's a simple workaround:

On your host generate a new ed25519 key

$ ssh-keygen -a 100 -t ed25519 -f ubuntu_vagrant -C vagrant

This will result in ubuntu_vagrant and ubuntu_vagrant.pub files.
2. vagrant up a new Ubuntu VM.
3. vagrant ssh into that VM. This will work because normal ssh client is used, not Ruby's net-ssh.
4. In the VM, put the previously generated ubuntu_vagrant.pub into ~/.ssh/authorized_keys. The original rsa key can be removed from ~/.ssh/authorized_keys.
5. On the host machine overwrite this VM's original private_key with ubuntu_vagrant

$ cp ubuntu_vagrant ${PATH_TO_VM_ROOT}/.vagrant/machines/default/virtualbox/private_key

Answer 4 · 2021-05-08T14:43:51.000Z

I've also tried it, but as OpenStack (at least in my case) is refusing this type of key, my only choice was RSA. An RSA key could be generated using SHA2, which should be ok for ssh, but not ok in Vagrant, because of the assumption that it will be SHA1. So... no possible workaround here, only a downgrade to 2.2.15 until it's fixed in 2.2.17. Le sam. 8 mai 2021 à 16:36, Sergey Glushchenko ***@***.***> a écrit :

…

I've run into this issue today as well with Ubuntu 21.04. Here's a simple workaround that worked for me: 1. On your host generate a ed25519 key $ ssh-keygen -a 100 -t ed25519 -f ubuntu_vagrant -C vagrant This will result in ubuntu_vagrant and ubuntu_vagrant.pub files. 2. vagrant up a new Ubuntu VM. 3. vagrant ssh into that VM. This will work because normal ssh client is used, not Ruby's net-ssh. 4. In the VM, put the previously generated ubuntu_vagrant.pub into ~/.ssh/authorized_keys. The original rsa key can be removed from ~/.ssh/authorized_keys. 5. On the host machine overwrite this VM's original private_key with ubuntu_vagrant cp ubuntu_vagrant ${PATH_TO_VM_ROOT}/.vagrant/machines/default/virtualbox/private_key — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12344 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACCUJBFX5UR2ARHBFQ5OLHLTMVD5HANCNFSM436QS2CQ> .

Answer 5 · 2021-05-11T20:31:02.000Z

Wanted to point out that AWS EC2 keypairs must be RSA (specifically 2048-bit SSH-2 RSA keys) so this change breaks the ability to use vagrant-aws to spin up EC2 boxes, IIUC.

Answer 6 · 2021-05-16T01:30:38.000Z

I just upgraded from Vagrant 2.2.14 to 2.2.16 and having the same issue.

Answer 7 · 2021-05-16T01:45:42.000Z

Downgrading back to Vagrant 2.2.14 solved it for me too.

Answer 8 · 2021-05-20T12:32:48.000Z

Here is a script to create the workaround, with credit to gsserge

EDIT: This will work only on systems that can use keys other than RSA.

#!/usr/bin/env bash

echo "Fixing SSH key for vagrant 2.2.16 ..."

# exit when any command fails
set -e
# keep track of the last executed command
trap 'last_command=$current_command; current_command=$BASH_COMMAND' DEBUG
# echo an error message before exiting
trap 'echo "\"${last_command}\" command filed with exit code $?."' EXIT

config="$(vagrant ssh-config)"
host="$(echo "$config" | grep "^  HostName" | cut -d' ' -f4)"
file="$(echo "$config" | grep "^  IdentityFile" | cut -d' ' -f4)"

[ -z "$host" ] || [ -z "$file" ] && echo "Failed to get vagrant config" && exit 1

echo "Generating key..."
cat /dev/zero | ssh-keygen -a 100 -t ed25519 -f ubuntu_vagrant -C vagrant -q -N '' || echo "Using existing key"

echo "Copying public key..."
ssh-copy-id -o "IdentityFile $file" -f -i ubuntu_vagrant.pub -p 22 vagrant@$host 1>/dev/null

echo "Copying private key..."
mv "$file" "$file.bak"
cp ubuntu_vagrant "$file"

echo "Testing connection..."
ssh -o 'BatchMode=yes' -o 'ConnectionAttempts=1' -i "$file" -q -p 22 vagrant@$host exit
echo "Connection successful"

echo "Removing temporary keys..."
rm ubuntu_vagrant*

trap 'echo "Success."' EXIT

set +e

Answer 9 · 2021-05-20T12:40:29.000Z

This fix doesn't apply to all situations. For my part, in my OpenStack infrastructure, I can only use RSA keys. Again, changing the type of key is not a fix, it's exactly following the rules forced by the new code, which simply forbid RSA keys for good. Le jeu. 20 mai 2021 à 14:33, Rob Record ***@***.***> a écrit :

…

Here is a script to create the workaround, with credit to gsserge #!/usr/bin/env bash echo "Fixing SSH key for vagrant 2.2.16 ..." # exit when any command failsset -e# keep track of the last executed commandtrap 'last_command=$current_command; current_command=$BASH_COMMAND' DEBUG# echo an error message before exitingtrap 'echo "\"${last_command}\" command filed with exit code $?."' EXIT config="$(vagrant ssh-config)" host="$(echo "$config" | grep "^ HostName" | cut -d' ' -f4)" file="$(echo "$config" | grep "^ IdentityFile" | cut -d' ' -f4)" [ -z "$host" ] || [ -z "$file" ] && echo "Failed to get vagrant config" && exit 1 echo "Generating key..." cat /dev/zero | ssh-keygen -a 100 -t ed25519 -f ubuntu_vagrant -C vagrant -q -N '' || echo "Using existing key" echo "Copying public key..." ssh-copy-id -o "IdentityFile $file" -f -i ubuntu_vagrant.pub -p 22 vagrant@$host 1>/dev/null echo "Copying private key..." mv "$file" "$file.bak" cp ubuntu_vagrant "$file" echo "Testing connection..." ssh -o 'BatchMode=yes' -o 'ConnectionAttempts=1' -i "$file" -q -p 22 vagrant@$host exitecho "Connection successful" echo "Removing temporary keys..." rm ubuntu_vagrant* trap 'echo "Success."' EXIT set +e — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12344 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACCUJBDYIL2N6JUUSIBOUKTTOT6QNANCNFSM436QS2CQ> .

Answer 10 · 2021-05-20T13:30:33.000Z

This fix doesn't apply to all situations.

Thanks; I have edited my comment to state it will only work where other keys are possible to use.

Answer 11 · 2021-06-03T19:59:12.000Z

I just wanted to add that I was also experiencing this issue when using vagrant-aws.
Vagrant would hang while trying to establish an SSH connection when executing vagrant up even though I could ssh to the host successfully.

I was able to uninstall and reinstall using version 2.2.15 with homebrew like this:

wget https://github.com/Homebrew/homebrew-cask/blob/d81815ea27a778a312fa0e2bbef0d78f9767f45b/Casks/vagrant.rb
brew install --cask vagrant.rb

Now everything works as expected.

Answer 12 · 2021-06-05T00:07:30.000Z

Hi everyone, sorry for the delay in this issue getting some attention. I'd like to take just a second to clarify some assertions that have been made in this issue since the naming of things ends up conflating two different things. The changes referenced in the net-ssh patch are not making any kinds of assumptions about the type of rsa key that is in use. Instead, the changes are focused on the signature schemes. With OpenSSH 8.6, the ssh-rsa signature scheme was fully deprecated and disabled. The result is that connections would no longer succeed where ssh-rsa signature scheme was being used with the RSA public key algorithm.

In OpenSSH 7.2 two new signature schemes were added, rsa-sha2-512 and rsa-sha2-256. These are what have now replaced ssh-rsa as of OpenSSH 8.6. The vagrant ssh command will happily work because it uses the ssh executable and it was updated for these changes quite awhile ago. Things like the provision command use the internal communicator, and this is what the patches are intended to fix.

Using @AntonioMeireles box (#12344 (comment)) I'm not having any issue connecting (using the current box, and one from a bit over a month ago). If anyone is encountering this issue using a publicly available box, please let me know as it would be very helpful to determine what's causing the issue.

Answer 13 · 2021-06-05T12:07:45.000Z

@chrisroberts

FWIW this is happening on macOS (BigSur - 11.4 (20F71)).

With plain 'empty' Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "AntonioMeireles/ClearLinux"
end

i get on vagrant up --debug --provider=virtualbox (vmware one is similar)

(...)
GuestAdditionsFacility_Graphics Mode=0,1622894219464
DEBUG subprocess: Waiting for process to exit. Remaining to timeout: 32000
DEBUG subprocess: Exit status: 0
DEBUG virtualbox_6_1:   - [1, "ssh", 2222, 22, "127.0.0.1"]
DEBUG ssh: Checking key permissions: /Users/am/.vagrant.d/insecure_private_key
 INFO ssh: Attempting SSH connection...
 INFO ssh: Attempting to connect to SSH...
 INFO ssh:   - Host: 127.0.0.1
 INFO ssh:   - Port: 2222
 INFO ssh:   - Username: clear
 INFO ssh:   - Password? false
 INFO ssh:   - Key Path: ["/Users/am/.vagrant.d/insecure_private_key"]
DEBUG ssh:   - connect_opts: {:auth_methods=>["none", "hostbased", "publickey"], :config=>false, :forward_agent=>false, :send_env=>false, :keys_only=>true, :verify_host_key=>:never, :password=>nil, :port=>2222, :timeout=>15, :user_known_hosts_file=>[], :verbose=>:debug, :logger=>#<Logger:0x00007f885993f7b8 @level=0, @progname=nil, @default_formatter=#<Logger::Formatter:0x00007f885993f740 @datetime_format=nil>, @formatter=nil, @logdev=#<Logger::LogDevice:0x00007f885993f6a0 @shift_period_suffix=nil, @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<StringIO:0x00007f885993f858>, @mon_mutex=#<Thread::Mutex:0x00007f885993f5d8>, @mon_mutex_owner_object_id=70111797574480, @mon_owner=nil, @mon_count=0>>, :keys=>["/Users/am/.vagrant.d/insecure_private_key"], :remote_user=>"clear"}
DEBUG ssh: == Net-SSH connection debug-level log START ==
DEBUG ssh: D, [2021-06-05T12:57:07.570460 #99269] DEBUG -- net.ssh.transport.session[3fc42cc9e304]: establishing connection to 127.0.0.1:2222
D, [2021-06-05T12:57:07.570813 #99269] DEBUG -- net.ssh.transport.session[3fc42cc9e304]: connection established
I, [2021-06-05T12:57:07.570860 #99269]  INFO -- net.ssh.transport.server_version[3fc42cca2ff8]: negotiating protocol version
D, [2021-06-05T12:57:07.570875 #99269] DEBUG -- net.ssh.transport.server_version[3fc42cca2ff8]: local is `SSH-2.0-Ruby/Net::SSH_6.1.0 x86_64-darwin19'
D, [2021-06-05T12:57:07.590389 #99269] DEBUG -- net.ssh.transport.server_version[3fc42cca2ff8]: remote is `SSH-2.0-OpenSSH_8.4'
W, [2021-06-05T12:57:07.590514 #99269]  WARN -- net.ssh.transport.algorithms[3fc42cca7788]: unsupported algorithm: `["rsa-sha2-256", "rsa-sha2-512"]'
I, [2021-06-05T12:57:07.590675 #99269]  INFO -- net.ssh.transport.algorithms[3fc42cca7788]: sending KEXINIT
D, [2021-06-05T12:57:07.590817 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 0 type 20 len 796
D, [2021-06-05T12:57:07.590871 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 800 bytes
D, [2021-06-05T12:57:07.591366 #99269] DEBUG -- socket[3fc42cca3ca0]: read 1056 bytes
D, [2021-06-05T12:57:07.591428 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 0 type 20 len 1052
I, [2021-06-05T12:57:07.591454 #99269]  INFO -- net.ssh.transport.algorithms[3fc42cca7788]: got KEXINIT from server
I, [2021-06-05T12:57:07.591531 #99269]  INFO -- net.ssh.transport.algorithms[3fc42cca7788]: negotiating algorithms
D, [2021-06-05T12:57:07.591663 #99269] DEBUG -- net.ssh.transport.algorithms[3fc42cca7788]: negotiated:
* kex: ecdh-sha2-nistp521
* host_key: ssh-ed25519
* encryption_server: aes256-ctr
* encryption_client: aes256-ctr
* hmac_client: hmac-sha2-512-etm@openssh.com
* hmac_server: hmac-sha2-512-etm@openssh.com
* compression_client: none
* compression_server: none
* language_client:
* language_server:
D, [2021-06-05T12:57:07.591694 #99269] DEBUG -- net.ssh.transport.algorithms[3fc42cca7788]: exchanging keys
D, [2021-06-05T12:57:07.595227 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 1 type 30 len 148
D, [2021-06-05T12:57:07.595291 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 152 bytes
D, [2021-06-05T12:57:07.605411 #99269] DEBUG -- socket[3fc42cca3ca0]: read 312 bytes
D, [2021-06-05T12:57:07.605504 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 1 type 31 len 292
D, [2021-06-05T12:57:07.609183 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 2 type 21 len 20
D, [2021-06-05T12:57:07.609242 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 24 bytes
D, [2021-06-05T12:57:07.609288 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 2 type 21 len 12
D, [2021-06-05T12:57:07.626136 #99269] DEBUG -- net.ssh.authentication.session[3fc42cccf350]: beginning authentication of `clear'
D, [2021-06-05T12:57:07.626205 #99269] DEBUG -- socket[3fc42cca3ca0]: using encrypt-then-mac
D, [2021-06-05T12:57:07.626266 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 3 type 5 len 32
D, [2021-06-05T12:57:07.626324 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 100 bytes
D, [2021-06-05T12:57:07.626826 #99269] DEBUG -- socket[3fc42cca3ca0]: read 100 bytes
D, [2021-06-05T12:57:07.626907 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 3 type 6 len 32
D, [2021-06-05T12:57:07.626969 #99269] DEBUG -- net.ssh.authentication.session[3fc42cccf350]: trying none
D, [2021-06-05T12:57:07.627037 #99269] DEBUG -- socket[3fc42cca3ca0]: using encrypt-then-mac
D, [2021-06-05T12:57:07.627083 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 4 type 50 len 48
D, [2021-06-05T12:57:07.627125 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 116 bytes
D, [2021-06-05T12:57:07.636259 #99269] DEBUG -- socket[3fc42cca3ca0]: read 116 bytes
D, [2021-06-05T12:57:07.636378 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 4 type 51 len 48
D, [2021-06-05T12:57:07.636431 #99269] DEBUG -- net.ssh.authentication.session[3fc42cccf350]: allowed methods: publickey,keyboard-interactive
D, [2021-06-05T12:57:07.636478 #99269] DEBUG -- net.ssh.authentication.methods.none[3fc436c0d9bc]: none failed
D, [2021-06-05T12:57:07.636509 #99269] DEBUG -- net.ssh.authentication.session[3fc42cccf350]: trying publickey
D, [2021-06-05T12:57:07.636949 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: connecting to ssh-agent
D, [2021-06-05T12:57:07.637059 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: sending agent request 1 len 47
D, [2021-06-05T12:57:07.637146 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: received agent packet 5 len 1
D, [2021-06-05T12:57:07.637171 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: sending agent request 11 len 0
D, [2021-06-05T12:57:07.637239 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: received agent packet 12 len 637
D, [2021-06-05T12:57:07.637484 #99269] DEBUG -- net.ssh.authentication.methods.publickey[3fc42d0cf8ec]: trying publickey (dd:3b:b8:2e:85:04:06:e9:ab:ff:a8:0a:c0:04:6e:d6)
D, [2021-06-05T12:57:07.637531 #99269] DEBUG -- net.ssh.authentication.methods.publickey[3fc42d0cf8ec]: public key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-06-05T12:57:07.637613 #99269] DEBUG -- socket[3fc42cca3ca0]: using encrypt-then-mac
D, [2021-06-05T12:57:07.637659 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 5 type 50 len 352
D, [2021-06-05T12:57:07.637703 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 420 bytes
D, [2021-06-05T12:57:07.638473 #99269] DEBUG -- socket[3fc42cca3ca0]: read 372 bytes
D, [2021-06-05T12:57:07.638555 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 5 type 60 len 304
D, [2021-06-05T12:57:07.638602 #99269] DEBUG -- net.ssh.authentication.methods.publickey[3fc42d0cf8ec]: public key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-06-05T12:57:07.638682 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: sending agent request 13 len 696
D, [2021-06-05T12:57:07.641707 #99269] DEBUG -- net.ssh.authentication.agent[3fc436c3c578]: received agent packet 14 len 276
D, [2021-06-05T12:57:07.641746 #99269] DEBUG -- net.ssh.authentication.methods.publickey[3fc42d0cf8ec]: public key has been marked for deprecated ssh-rsa SHA1 behavior
D, [2021-06-05T12:57:07.641813 #99269] DEBUG -- socket[3fc42cca3ca0]: using encrypt-then-mac
D, [2021-06-05T12:57:07.641863 #99269] DEBUG -- socket[3fc42cca3ca0]: queueing packet nr 6 type 50 len 624
D, [2021-06-05T12:57:07.641905 #99269] DEBUG -- socket[3fc42cca3ca0]: sent 692 bytes
D, [2021-06-05T12:57:07.655395 #99269] DEBUG -- socket[3fc42cca3ca0]: read 116 bytes
D, [2021-06-05T12:57:07.655514 #99269] DEBUG -- socket[3fc42cca3ca0]: received packet nr 6 type 51 len 48
D, [2021-06-05T12:57:07.655562 #99269] DEBUG -- net.ssh.authentication.session[3fc42cccf350]: allowed methods: publickey,keyboard-interactive
D, [2021-06-05T12:57:07.655592 #99269] DEBUG -- net.ssh.authentication.methods.publickey[3fc42d0cf8ec]: publickey failed (dd:3b:b8:2e:85:04:06:e9:ab:ff:a8:0a:c0:04:6e:d6)
E, [2021-06-05T12:57:07.655621 #99269] ERROR -- net.ssh.authentication.session[3fc42cccf350]: all authorization methods failed (tried none, publickey)

DEBUG ssh: == Net-SSH connection debug-level log END ==
 INFO ssh: SSH not ready: #<Vagrant::Errors::SSHAuthenticationFailed: SSH authentication failed! This is typically caused by the public/private
keypair for the SSH user not being properly set on the guest VM. Please
verify that the guest VM is setup with the proper public key, and that
the private key path for Vagrant is setup properly as well.>
(...)

i can, as everybody else, after the up fails, ssh to the box via vagrant ssh (which consumes system ssh client).

It's just that the initial provisioning (done at box creation or box power up) which must be done via the internal vagrant' s ssh communicator won't be done anymore any as the internal ssh communicator just stopped behaving :-/ [which breaks obviously all sorts of automations/ CI that one had :-( ]

Hope this helps!

All the Best!

António

Answer 14 · 2021-06-05T17:58:52.000Z

@chrisroberts

I'd like to take just a second to clarify some assertions that have been made in this issue since the naming of things ends up conflating two different things.

Thanks very much for the clarification! I hadn't actually heard of signature schemes before. xD

If anyone is encountering this issue using a publicly available box, please let me know as it would be very helpful to determine what's causing the issue.

I put together a reproduction repo that I hope you'll find helpful. You'll need an AWS account to use it but I also provided logs from my own run.

Some interesting findings:

aws ec2 create-key-pair generated keys work with Vagrant 2.2.16. They're 2048 bit RSA keys, which is smaller than what I advocate my team uses which is 4096. I wasn't able to find how to check what signature scheme a key uses at this point. If you know how to do that it would be helpful to me.
Neither ssh-keygen -t rsa or ssh-keygen -t rsa-sha2-512 generated keys that were usable by Vagrant 2.2.16. Again, these keys work from ssh and from vagrant ssh, just not from the internal communicator. All this is visible in the logs in the gist linked from the repo. I must be misunderstanding the ssh-keygen man page but I'm not able to find any more information at the moment about how to generate these keys. My team needs to generate the keys outside of EC2 for a variety reasons, though, so using the aws ec2 keypair isn't tenable without generating a lot of additional work.

Let me know if there's anything else I can do to help.

Answer 15 · 2021-06-10T17:47:22.000Z

@timvisher Thanks so much for putting together that production repository. It was extremely helpful and has allowed me to reproduce the behavior and locate the source of the problem. Wanted to leave an update just to let everyone know this is being worked on, it's just been a busy week. I'm hoping I'll be able to put together a fix and get a PR up at some point today.

Answer 16 · 2021-06-11T16:37:21.000Z

Linked PR has updated patch that resolves the connection issues. Will be included in the next release.

Answer 17 · 2021-06-17T21:44:00.000Z

@chrisroberts sorry but I think this is the rong way,
As it will only work with openssh and miss all the other ssh server Denons some are listed here https://en.m.wikipedia.org/wiki/Comparison_of_SSH_servers

This maybe will help
http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.bin/ssh/PROTOCOL.certkeys?content-type=text/plain
"These RSA/SHA-2 types should not appear in keys at rest or transmitted
on the wire, but do appear in a SSH_MSG_KEXINIT's host-key algorithms
field or in the "public key algorithm name" field of a "publickey"
SSH_USERAUTH_REQUEST to indicate that the signature will use the
specified algorithm."

Answer 18 · 2021-06-27T13:27:08.000Z

Just a note on the script (thank you!) by @robrecord, I had to use the port from vagrant ssh-config not a hard-coded port 22.

port="$(echo "$config" | grep "^  Port" | cut -d' ' -f4)"
# ...
ssh-copy-id -o "IdentityFile $file" -f -i ubuntu_vagrant.pub -p $port vagrant@$host 1>/dev/null
# ...
ssh -o 'BatchMode=yes' -o 'ConnectionAttempts=1' -i "$file" -q -p $port vagrant@$host exit

Answer 19 · 2021-07-28T02:09:35.000Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.