wait_for_connection times out when using Ansible with mitogen on standard Rocky 8.9 AWS images

[pstrzelczak]

When running wait_for_connection Ansible task with mitogen 0.3.4 on AWS EC2 instance it always times out and produces the following output:

│ TASK [wait_for_connection]
│ *****************************************************
│ task path:
│ YYY:4
│ Tuesday 12 March 2024  11:20:16 +0000 (0:00:00.051)       0:00:00.051
│ ********* 
│ [WARNING]: Unhandled error in Python interpreter discovery for host
│ XXX: EOF on stream; last 100 lines
│ received: ssh: connect to host XXX
│ port
│ 22: Connection refused
│ fatal: [XXX]: FAILED! => {"changed":
│ false, "elapsed": 600, "msg": "timed out waiting for ping module test: EOF
│ on stream; last 100 lines received:\nbash: /usr/bin/python: No such file or
│ directory"}
│ 
│ NO MORE HOSTS LEFT

I also checked that executing wait_for_connection module with mitogen after ssh become available works file.
Detailed logs seem not to add much more information because it seems after unhandled exception in python discovery due to not yet ready ssh server, the code just loops and keeps failing because it somehow expects /usr/bin/python to be there.

• Which version of Ansible are you running?

$ ansible --version
ansible [core 2.13.13]
  config file = None
  configured module search path = ['/home/docker_user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/docker_user/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.17 (default, Aug 10 2023, 18:03:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]
  jinja version = 3.1.3
  libyaml = True

• Is your version of Ansible patched in any way?
  No

• Are you running with any custom modules, or module_utils loaded?
  Yes, but they should not be relevant here.

• Have you tried the latest master version from Git?
  No

• Do you have some idea of what the underlying problem may be?
  https://mitogen.networkgenomics.com/ansible_detailed.html#common-problems has
  instructions to help figure out the likely cause and how to gather relevant
  logs.
  No

• Mention your host and target OS and versions
  Rocky 8.9 on both

• Mention your host and target Python versions
  Python 3.8 on host and platform python on target (no Python installed there)

• If reporting any kind of problem with Ansible, please include the Ansible
  version along with output of "ansible-config dump --only-changed".
  Nothing