testssl/testssl.sh

[BUG / possible BUG]: No communication of bailing early on compatible ciphers when server preference is invalid

Closed this issue ยท 17 comments

UPDATE: Please skip to follow-up comment, the original reproduction is flawed: #2883 (comment)


Before you open an issue please check which version you are running and whether it is the latest in stable / dev branch

GHCR.io 3.3dev image (published 3 hours ago)

Before you open an issue please whether this is a known problem by searching the issues

I couldn't find anything

Command line / docker command to reproduce

docker compose up -d --force-recreate
docker compose run --rm -it testssl --quiet --preference mail.example.test:993
Output
Testing all IPv4 addresses (port 993): 172.18.0.4
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Start 2025-09-06 08:56:03        -->> 172.18.0.4:993 (mail.example.test) <<--

 rDNS (172.18.0.4):      example-dms-1.example_default.
 Service detected:       IMAP, thus skipping HTTP specific checks

 Testing server's cipher preferences

Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits     Cipher Suite Name (IANA/RFC)
-----------------------------------------------------------------------------------------------------------------------------
SSLv2
 -
SSLv3
 -
TLSv1
 -
TLSv1.1
 -
TLSv1.2 (server order)
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 253   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 253   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
TLSv1.3 (server order)
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128      TLS_AES_128_GCM_SHA256

 Has server cipher order?     yes (OK) -- TLS 1.3 and below



 Done 2025-09-06 08:56:22 [  27s] -->> 172.18.0.4:993 (mail.example.test) <<--

------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Done testing now all IP addresses (on port 993): 172.18.0.4

Associated compose.yaml:

`compose.yaml` (collapsed for brevity)
name: example

services:
  testssl:
    scale: 0 # This avoids running the container during `docker compose up`
    image: ghcr.io/testssl/testssl.sh:3.3dev
    # Config for `testssl.sh` to trust the locally generated cert:
    environment:
      ADDTL_CA_FILES: "/tmp/tls/ca-cert.pem"
    configs:
      - source: tls-ca-cert
        target: /tmp/tls/ca-cert.pem

  # Service to test against:
  dms:
    image: mailserver/docker-mailserver:15.1.0
    hostname: mail.example.test
    environment:
      - SSL_TYPE=manual
      - SSL_KEY_PATH=/tmp/tls/key.pem
      - SSL_CERT_PATH=/tmp/tls/cert.pem
    configs:
      # Extra config bundled into the `compose.yaml` to ease testing:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf
      - source: tls-cert
        target: /tmp/tls/cert.pem
      - source: tls-key
        target: /tmp/tls/key.pem
      # Required to reproduce the observed bug by removing DHE support (while retaining DHE in server order):
      - source: dovecot-overrides
        target: /tmp/docker-mailserver/dovecot.cf
      # Optional - Allows clients in the container to verify cert trust with the CA that signed it:
      - source: tls-ca-cert
        target: /tmp/tls/ca-cert.pem

configs:
  # Pre-provisioned account for local reproduction use:
  # NOTE: `$` is escaped by repeating it to avoid the Docker Compose ENV interpolation feature
  # Password is `secret`
  dms-accounts:
    content: |
      john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.

  # Example ECDSA cert files for testing locally (provisioned for `mail.example.test`):
  tls-ca-cert:
    content: |
      -----BEGIN CERTIFICATE-----
      MIIBfTCCASKgAwIBAgIRAMAZttlRlkcuSun0yV0z4RwwCgYIKoZIzj0EAwIwHDEa
      MBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcNMzEw
      MTAxMDAwMDAwWjAcMRowGAYDVQQDExFTbWFsbHN0ZXAgUm9vdCBDQTBZMBMGByqG
      SM49AgEGCCqGSM49AwEHA0IABJX2hCtoK3+bM5I3rmyApXLJ1gOcVhtoSSwM8XXR
      SEl25Kkc0n6mINuMK8UrBkiBUgexf6CYayx3xVr9TmMkg4KjRTBDMA4GA1UdDwEB
      /wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/AgEBMB0GA1UdDgQWBBQD8sBrApbyYyqU
      y+/TlwGynx2V5jAKBggqhkjOPQQDAgNJADBGAiEAi8N2eOETI+6hY3+G+kzNMd3K
      Sd3Ke8b++/nlwr5Fb/sCIQDYAjpKp/MpTDWICeHC2tcB5ptxoTdWkTBuG4rKcktA
      0w==
      -----END CERTIFICATE-----

  tls-key:
    content: |
      -----BEGIN EC PRIVATE KEY-----
      MHcCAQEEIOc6wqZmSDmT336K4O26dMk1RCVc0+cmnsO2eK4P5K5yoAoGCCqGSM49
      AwEHoUQDQgAEFOWNgekKKvUZE89vJ7henUYxODYIvCiHitRc2ylwttjqt1KUY1cp
      q3jof2fhURHfBUH3dHPXLHig5V9Jw5gqeg==
      -----END EC PRIVATE KEY-----

  tls-cert:
    content: |
      -----BEGIN CERTIFICATE-----
      MIIB9DCCAZqgAwIBAgIQE53a/y2c//YXRsz2kLm6gDAKBggqhkjOPQQDAjAcMRow
      GAYDVQQDExFTbWFsbHN0ZXAgUm9vdCBDQTAeFw0yMTAxMDEwMDAwMDBaFw0zMTAx
      MDEwMDAwMDBaMBkxFzAVBgNVBAMTDlNtYWxsc3RlcCBMZWFmMFkwEwYHKoZIzj0C
      AQYIKoZIzj0DAQcDQgAEFOWNgekKKvUZE89vJ7henUYxODYIvCiHitRc2ylwttjq
      t1KUY1cpq3jof2fhURHfBUH3dHPXLHig5V9Jw5gqeqOBwDCBvTAOBgNVHQ8BAf8E
      BAMCB4AwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMB0GA1UdDgQWBBSz
      w74g+O6dcBbwienD70D8A9ESmDAfBgNVHSMEGDAWgBQD8sBrApbyYyqUy+/TlwGy
      nx2V5jBMBgNVHREERTBDghFtYWlsLmV4YW1wbGUudGVzdIIVbWFpbC5kZXN0aW5h
      dGlvbi50ZXN0ghdzbXRwLnJlbGF5LXNlcnZpY2UudGVzdDAKBggqhkjOPQQDAgNI
      ADBFAiEAoety5oClZtuBMkvlUIWRmWlyg1VIOZ544LSEbplsIhcCIHb6awMwNdXP
      m/xHjFkuwH1+UjDDRW53Ih7KZoLrQ6Cp
      -----END CERTIFICATE-----

  dovecot-overrides:
    content: |
      # Unset this setting to no longer support negotiating DHE cipher suites
      ssl_dh =

      # This setting is the equivalent value configured in the image,
      # except the DHE cipher suites have been removed. Uncomment this setting to get expected output:
      # Ref: https://github.com/docker-mailserver/docker-mailserver/blob/5f4e868c541b142c705a643315ba8828d1836660/target/scripts/helpers/ssl.sh#L145-L146
      #ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384

Expected behavior

This took a while to troubleshoot, but eventually realized OpenSSL could negotiate it successfully (EDIT: Bad recall on my part), but testssl.sh was failing to report the cipher, I have found I could successfully negotiate the cipher suites that are expected to remain compatible:

# Directly verify `-cipher` negotiates successfully (requires uncommenting `ssl_cipher_list` in `compose.yaml`):
$ docker compose exec dms bash
$ timeout 1 openssl s_client -tls1_2 -cipher ECDHE-ECDSA-AES256-SHA384 -connect mail.example.test:993 -CAfile /tmp/tls/ca-cert.pem

UPDATE: I thought this was a testssl.sh issue, but realized while filing the report that my OpenSSL results were actually with the corrected Dovecot ssl_cipher_list setting (EDIT: this observation was flawed, OpenSSL can directly negotiate a compatible cipher successfully) ๐Ÿ˜“ Assuming that testssl.sh behaves the same as OpenSSL, it may not be able to negotiate any other compatible cipher due to the configured server preference order for Dovecot (which is where the actual bug of a contribution was coming from, caught thanks to testssl.sh! โค๏ธ).


Instead of this incorrect result:

TLSv1.2 (server order)
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 253   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 253   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256

I expected the valid ciphers (or some warning emitted about bailing early):

TLSv1.2 (server order)
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 253   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 253   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xc023   ECDHE-ECDSA-AES128-SHA256         ECDH 253   AES         128      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
 xc024   ECDHE-ECDSA-AES256-SHA384         ECDH 253   AES         256      TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384

or equivalent if results were stored via JSON using options --jsonfile-pretty port_993.json --overwrite:

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Get the TLS 1.2 cipher suite server order:
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipherorder_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-ECDSA-CHACHA20-POLY1305 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES256-SHA384

Your system (please complete the following information):

  • N/A (using the GHCR.io image)

Additional context

--debug 2 comparison:

Actual: (bailed early on (TLS) ServerHello empty for failed negotiation on DHE ciphersuite?)

Output
TLSv1.2
sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
(TLS) ServerHello empty, TCP connection closed
  (1 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)
 (server order)
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 253   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 253   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256

Expected:

Output
TLSv1.2
sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
sending close_notify...
  (46 lines returned)

sending client hello... sending client hello... reading server hello...
(TLS) ServerHello empty, TCP connection closed
  (1 lines returned)
 (server order)
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 253   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 253   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xc023   ECDHE-ECDSA-AES128-SHA256         ECDH 253   AES         128      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
 xc024   ECDHE-ECDSA-AES256-SHA384         ECDH 253   AES         256      TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384

Closing as it looks like there might not be anything testssl.sh can do here to improve the UX and warn about a server claiming to support a cipher suite, then failing to negotiate that with server preference preventing access to actual compatible ciphers?

Actually no... I'm double mistaken ๐Ÿ˜“ For the reproduction I simplified this with a report with ECDSA as the cert, but the ssl_cipher_list setting in the compose.yaml was actually for the DMS setting TLS_LEVEL=intermediate. Hence these additional ciphersuites were not actually permitted/compatible with the default TLS_LEVEL=modern setting in DMS, thus OpenSSL failed to negotiate like testssl.sh.

My earlier experience of OpenSSL negotiating the cipher suite directly was successful while testssl.sh bailed early instead of listing the compatible cipher suite when DMS used TLS_LEVEL=intermediate with an RSA certificate. Uncommenting the ssl_cipher_list setting in compose.yaml then makes testssl.sh successful.

So now I am wondering if it is a bug in testssl.sh, and if it could detect this? Or given the service tested (Dovecot) is misleading with cipher suites it supports due to misconfiguration, is the behaviour of testssl.sh correct here for server preference? Or is there information that testssl.sh can use to better communicate this problem when it exists?


Reproduction with RSA cert + DMS (with TLS_LEVEL=intermediate)

Expectation was testssl.sh would convey compatible ciphers, but perhaps that requires bluntly testing with --each-cipher?:

`-e, --each-cipher` checks each of the (currently configured) 370 ciphers via openssl + sockets remotely on the server and reports back the result in wide mode. If you want to display each cipher tested you need to add `--show-each`. Per default it lists the following parameters: `hexcode`, `OpenSSL cipher suite name`, `key exchange`, `encryption bits`, `IANA/RFC cipher suite name`. Please note the `--mapping` parameter changes what cipher suite names you will see here and at which position. Also please note that the __bit__ length for the encryption is shown and not the __security__ length, albeit it'll be sorted by the latter. For 3DES due to the Meet-in-the-Middle problem the bit size of 168 bits is equivalent to the security size of 112 bits.

EDIT: --each-cipher was no better.


`compose.yaml` (click to view)
name: example

services:
  testssl:
    scale: 0 # This avoids running the container during `docker compose up`
    image: ghcr.io/testssl/testssl.sh:3.3dev
    # Config for `testssl.sh` to trust the locally generated cert:
    environment:
      ADDTL_CA_FILES: "/tmp/tls/ca-cert.pem"
    configs:
      - source: tls-ca-cert
        target: /tmp/tls/ca-cert.pem

  # Service to test against:
  dms:
    image: mailserver/docker-mailserver:15.1.0
    hostname: mail.example.test
    environment:
      - SSL_TYPE=manual
      - SSL_KEY_PATH=/tmp/tls/key.pem
      - SSL_CERT_PATH=/tmp/tls/cert.pem
      - TLS_LEVEL=intermediate
    configs:
      # Extra config bundled into the `compose.yaml` to ease testing:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf
      - source: tls-cert
        target: /tmp/tls/cert.pem
      - source: tls-key
        target: /tmp/tls/key.pem
      # Required to reproduce the observed bug by removing DHE support (while retaining DHE in server order):
      - source: dovecot-overrides
        target: /tmp/docker-mailserver/dovecot.cf
      # Optional - Allows clients in the container to verify cert trust with the CA that signed it:
      - source: tls-ca-cert
        target: /tmp/tls/ca-cert.pem

configs:
  # Pre-provisioned account for local reproduction use:
  # NOTE: `$` is escaped by repeating it to avoid the Docker Compose ENV interpolation feature
  # Password is `secret`
  dms-accounts:
    content: |
      john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.

  # Example RSA cert files for testing locally (provisioned for `mail.example.test`):
  tls-ca-cert:
    content: |
      -----BEGIN CERTIFICATE-----
      MIIDCDCCAfCgAwIBAgIRANPCP+3+4OKj3S2gIsfQcoIwDQYJKoZIhvcNAQELBQAw
      HDEaMBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcN
      MzEwMTAxMDAwMDAwWjAcMRowGAYDVQQDExFTbWFsbHN0ZXAgUm9vdCBDQTCCASIw
      DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKuYF7paXiAvIK8IIE6rnNMSuS94
      z8+1md2a6DnijBiwG/DcB2e1wxBab/kK+XNsPWI26FLUxqc9zGZ0Nn7s7DOT6C52
      43m4vJUrTthsFdLzW9SFCbMFCikYhbOSqfj06oVHRstkAI+WgyX6QrnrSV+h0imk
      gZlwjk+oKpkE6rS8zCGgXZMqi7/06qiTwXCVStMnEnX9FuNTbMjgz6bVQH6FOE9e
      8QbykSUjWvE+x6Rk5gUpbFGV26Bb62zH6zF9FY6ECsdsJJlALJ7e5w6MTfSL23u8
      cRcBhtBaQuHkFg9G6JFigQwbm0DQf7o7crdMJY/paWbCuRhxXx3YGZFOU1sCAwEA
      AaNFMEMwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQIMAYBAf8CAQEwHQYDVR0O
      BBYEFB3P7cwUTP3cQLjy3SvvnSYQLz/DMA0GCSqGSIb3DQEBCwUAA4IBAQBcSu9x
      eGVGMjfxOmX1S+MyMJNPWKJuBlW5cX80h7mBIVIKnamJjMBDGu8WJFnaFdch5uwr
      KIlXbDmFJjxBb3id/6W3BC09Ze6fjtY+rPRPkCi5L5CbLs6KU1V2X/xoMhg15upH
      e29Giz2gK/Zd9/ewxa+sNRx2mk9fhIzgLA55q34OkhEdi0dJFlyzNHw+/6eYICPd
      hPv+8z3dreuyXqKBO+M/QqLXHqonvZWbJzeWCqetZn/h9WFM2WXMzRTR/GjaUIQP
      /ZPLXm5P4YMtAEbG+PfHuzC0bwUmj5FTGJ7I6v9Y32KS+P94KlsmHOJICqSwLQWo
      6MbojAT5BXYzHzHq
      -----END CERTIFICATE-----

  tls-key:
    content: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEpAIBAAKCAQEA4BRlXM4r0CmZyN+rnH2jjwRAuZmb9duFtZAZXT3fIh5yxYkP
      dHH8Czl+jhMt3NLxdqjp2VerOVVE2O8KY8BOzp0mdU/W2b9qp5UXAoeqHmWEerH3
      cVgpM9sxylAcZ9U1jysPf9outNtxaKoZ33CspQBQk7ws5LgeEWyNxRjC6VeSiweJ
      iKhkiaSyrVn9y7Uvd/NS48flbJGTw2H5pdS5ekcbFNLV2NKnt3c6MS8rk5vwAlCO
      RjdVL3QqXGpD3h6k9c/GnSF1ik7FuzBdTQUwXhJ47XJz9BYp6oq08y9HCD4hfCRM
      eoNHJkwefnGu8A5pXWOSn6aMOm2hClQrMHQ5dQIDAQABAoIBAH6Yx1ORX4txlWK5
      i1kUWm2Yd4DkWgqjBX24dbwKEqBSF4Gml8awBzfIOcnG6ChUPPtPHx8duqzfkdAF
      7RbCNUPh4TJx8u5+iKE5SBCz2Nbnf9tZ5HRy3IRhmFW2wPWgrWu/ZjhTagPf3sjF
      IWztWXy3Gs78h1iI9OPfMpFiFeyB8LEE1w4nb/iSSUvFue/VZ0aDS5eOmUFVzjxy
      xNfHZ33QnmRYbsJ58oF2Fr+3fTmltM406y+Tg3+Dao+Mpf41NKq+1r58PnKpUjfH
      cLO5k4/Mqbnk4xi/ftzApIew1A6ClO9xwA3+oYE/S0LQ8JHEVajG1aMw8PEXWbOJ
      wzcXwEECgYEA7NZQDQHjCCzdmECITSr3I3tuXbmnv29RS4sAepn9fS4OVosnntrc
      YnRGriuFIZpyrr/3R5DwTgO61VeBAqvJDYtVjj6zyT9f87fdm8gJeDW7T0G24vt6
      0H4KG8ws8+I/FjMj7wiC4yN4/Hcyd94squh+/9NjdTvwfvS6frwnQR0CgYEA8jXV
      SRfBYJ8U8VBMZxfOmvEtJfxaahtnJGBBEwDcCfgB+rIjVQGw0aZONPKilD8jinAt
      UiBwyBDkHaUL14s6H23+QS6am6Mr9lTz+YPqpYWG0VjGBv8kuHZ50EhlBJFadzMb
      VVR5R5FDP3ChMQy7Q6e8RoIK5DQD86vKjQRegjkCgYAlERGsR3xR3ju8RXVPpobR
      bdMDJjhj1LdDfHjRt2IeAmRKFTNZQGW3nv0k6zjF3pdOVEsOT1fczeai1zQgx+QK
      k6ELRzL6L0oEKeWsKO2ae8ZaDC3kbnl1QhSw7w6mCOXYwp5AHfPmOroHwVwLuKED
      Cqo9vcbWJVBpfkHl7eqy3QKBgQCHOZDrbuzSsd4yX79YK0146b9oHryn0sbB409R
      ecBffGw2d7AMLJZ4Zd3x56jnFV0VVE2pNV1iBTQmbNfwrdV0aKdz4r4EuJO5wnI3
      0vN1F9hOFr7wdxAcQGD/7PshErmsJQdUm4Xec/ZUe+Ayj0YZnpMZ1k6YW4X9S+MY
      2eCd2QKBgQDQRh7HHS3n7PAwwtUDKR9oEFWE0dLOr+RWSeFsXz10gxUbQuHvLXU4
      nOCGA+Add6995PAs9xcnA/Ewju/l3YBgiQPvmLqEnpSZUiRoKQcTE+sAd9FCNB/4
      lORhokK5KlIkxUQISAZ65p0awQe0yyfOD1VJmcvALF1FRwWyz5M1TA==
      -----END RSA PRIVATE KEY-----

  tls-cert:
    content: |
      -----BEGIN CERTIFICATE-----
      MIIDXzCCAkegAwIBAgIRAKx4AavefugBgIJG3zel3P4wDQYJKoZIhvcNAQELBQAw
      HDEaMBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcN
      MzEwMTAxMDAwMDAwWjAZMRcwFQYDVQQDEw5TbWFsbHN0ZXAgTGVhZjCCASIwDQYJ
      KoZIhvcNAQEBBQADggEPADCCAQoCggEBAOAUZVzOK9Apmcjfq5x9o48EQLmZm/Xb
      hbWQGV093yIecsWJD3Rx/As5fo4TLdzS8Xao6dlXqzlVRNjvCmPATs6dJnVP1tm/
      aqeVFwKHqh5lhHqx93FYKTPbMcpQHGfVNY8rD3/aLrTbcWiqGd9wrKUAUJO8LOS4
      HhFsjcUYwulXkosHiYioZImksq1Z/cu1L3fzUuPH5WyRk8Nh+aXUuXpHGxTS1djS
      p7d3OjEvK5Ob8AJQjkY3VS90KlxqQ94epPXPxp0hdYpOxbswXU0FMF4SeO1yc/QW
      KeqKtPMvRwg+IXwkTHqDRyZMHn5xrvAOaV1jkp+mjDptoQpUKzB0OXUCAwEAAaOB
      njCBmzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUF
      BwMCMB0GA1UdDgQWBBQGnpb0nk82jVI3JImnfdHR7DYjADAfBgNVHSMEGDAWgBQd
      z+3MFEz93EC48t0r750mEC8/wzAqBgNVHREEIzAhggxleGFtcGxlLnRlc3SCEW1h
      aWwuZXhhbXBsZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQAKNre3khwoMoL0LoQG
      UF8HVi5OxzzKCzueZQMU8fj/MXQhpPWAkXKa4vXwCDvVRBCC1j4u+xSeiqXvMVcR
      n7QyxvYKJRnQ0k/x1zp8N6eed2tOFOz+gyHkoNSr/l9fQsAhqoL0FeVatqczI0co
      DGg5ux5bjZwllFYw6LRIuhtZ4BxIQO4GC5pysrvjXb782v0iAowQHL3yC0x/Eyfr
      ca/qovVST8zoWYf/1pQ/7Kp9do8VraB4dyr5r/zAy0GHPfia8qL864wTqGcuRnW3
      1y2BOcKLgnTNCFp2ncWodmObsxom8KgUAVyW06cDx4XI1wa2FSx4G5mehBgSd/mQ
      1Avy
      -----END CERTIFICATE-----

  dovecot-overrides:
    content: |
      # Unset this setting to no longer support negotiating DHE cipher suites
      ssl_dh =

      # This setting is the equivalent value configured in the image,
      # except the DHE cipher suites have been removed. Uncomment this setting to get expected output:
      # Ref: https://github.com/docker-mailserver/docker-mailserver/blob/5f4e868c541b142c705a643315ba8828d1836660/target/scripts/helpers/ssl.sh#L145-L146
      #ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384
docker compose up -d --force-recreate

docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993
# Actual:
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipherorder_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384

# Expected (uncommented `ssl_cipher_list` line in `compose.yaml`):
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipherorder_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-ECDSA-CHACHA20-POLY1305 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES256-SHA384
Output - Actual
Testing all IPv4 addresses (port 993): 172.18.0.4
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Start 2025-09-06 10:40:49        -->> 172.18.0.4:993 (mail.example.test) <<--

 rDNS (172.18.0.4):      example-dms-1.example_default.
 Service detected:       IMAP, thus skipping HTTP specific checks

 Testing server's cipher preferences

Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits     Cipher Suite Name (IANA/RFC)
-----------------------------------------------------------------------------------------------------------------------------
SSLv2
 -
SSLv3
 -
TLSv1
 -
TLSv1.1
 -
TLSv1.2 (server order)
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH 253   ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
 xc02f   ECDHE-RSA-AES128-GCM-SHA256       ECDH 253   AESGCM      128      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 xc030   ECDHE-RSA-AES256-GCM-SHA384       ECDH 253   AESGCM      256      TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLSv1.3 (server order)
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128      TLS_AES_128_GCM_SHA256

 Has server cipher order?     yes (OK) -- TLS 1.3 and below



 Done 2025-09-06 10:41:07 [  25s] -->> 172.18.0.4:993 (mail.example.test) <<--

------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Done testing now all IP addresses (on port 993): 172.18.0.4
Output - Expected
Testing all IPv4 addresses (port 993): 172.18.0.4
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Start 2025-09-06 10:42:54        -->> 172.18.0.4:993 (mail.example.test) <<--

 rDNS (172.18.0.4):      example-dms-1.example_default.
 Service detected:       IMAP, thus skipping HTTP specific checks

 Testing server's cipher preferences

Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits     Cipher Suite Name (IANA/RFC)
-----------------------------------------------------------------------------------------------------------------------------
SSLv2
 -
SSLv3
 -
TLSv1
 -
TLSv1.1
 -
TLSv1.2 (server order)
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH 253   ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
 xc02f   ECDHE-RSA-AES128-GCM-SHA256       ECDH 253   AESGCM      128      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 xc030   ECDHE-RSA-AES256-GCM-SHA384       ECDH 253   AESGCM      256      TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
 xc027   ECDHE-RSA-AES128-SHA256           ECDH 253   AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
 xc028   ECDHE-RSA-AES256-SHA384           ECDH 253   AES         256      TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
TLSv1.3 (server order)
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128      TLS_AES_128_GCM_SHA256

 Has server cipher order?     yes (OK) -- TLS 1.3 and below



 Done 2025-09-06 10:43:16 [  29s] -->> 172.18.0.4:993 (mail.example.test) <<--

------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Done testing now all IP addresses (on port 993): 172.18.0.4

Regardless of the before/after for ssl_cipher_list setting in compose.yaml below, the following is successful with OpenSSL:

# Directly verify `-cipher` negotiates successfully:
$ docker compose exec dms bash

$ timeout 1 openssl s_client -tls1_2 -cipher ECDHE-RSA-AES256-SHA384 -connect mail.example.test:993 -CAfile /tmp/tls/ca-cert.pem

I am inclined to close the issue as not a bug? (seems in my follow-up comment there is some unexplained behaviour though, but largely a misconfigured server is to blame for the results)

Given behaviour reported with OpenSSL directly below, but I would like confirmation on testssl.sh expectation to negotiate the other ciphersuites that should be compatible via --each-cipher?

Is that not doing the equivalent of looping through each ciphersuite to call -cipher "${CIPHERSUITE_HERE}"? Why would it bail early still? When using OpenSSL that way as shown in the previous comment, it would be successful. So I'd expect testssl.sh should be able to report the remaining compatible ciphersuites? (EDIT: Seems rather than query each cipher directly, it may instead be querying with each prior successful negotiation negated?)

For example the --each-cipher --show-all output filtered to a compatible ciphersuite claims it's not available for some reason?:

 xc027   ECDHE-RSA-AES128-SHA256           ECDH       AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256              not a/v

The ability to filter ciphers queried more specifically seems rather limited, no equivalent to OpenSSL -cipher pattern, only -x/--single-cipher pattern matching (which from the docs doesn't seem to support exclusion/negation or | conditions).


As noted in the previous comment, due to the Dovecot service reporting DHE cipher suites for server preference which are not actually supported due to lack of DHE params file, their position in ssl_cipher_list causes all subsequent cipher suites that would be compatible to be ignored when the client attempts to negotiate the assumed compatible DHE ciphersuite.

I'm not quite sure if this is a bug on testssl.sh, but I assume it's logic is to negotiate a connection, then repeat the connection each time with the previous ciphersuite negotiated added to a list of cipher suites to exclude? Then once the connection fails it is deemed complete. Hence testssl.sh lacks the information to know any better and bails due to the misconfigured service (unexpected when using --each-cipher however?).


Using OpenSSL to simulate described behaviour, we can exclude the reported ciphersuites from testssl.sh:

ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384

while including ALL in -cipher to permit negotiating any other cipher-suite.

This restricts what cipher suites the client will negotiate with, so should reflect something similar to what testssl.sh is doing (I'd try inspect the source script, but that's not too friendly given all source is bundled into a single file for some reason, would make more sense as modular snippets for development that are only concatenated together for distribution/execution?).

$ timeout 1 openssl s_client \
  -tls1_2 \
  -cipher 'ALL:!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

CONNECTED(00000003)
40E7B0B1797C0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:../ssl/record/rec_layer_s3.c:322:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 303 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1757209692
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

Allowing one of the reported cipher suites will however connect successfully:

$ timeout 1 openssl s_client \
  -tls1_2 \
  -cipher 'ALL:!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

CONNECTED(00000003)
depth=1 CN = Smallstep Root CA
verify return:1
depth=0 CN = Smallstep Leaf
verify return:1
---
Certificate chain
 0 s:CN = Smallstep Leaf
   i:CN = Smallstep Root CA
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jan  1 00:00:00 2021 GMT; NotAfter: Jan  1 00:00:00 2031 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDXzCCAkegAwIBAgIRAKx4AavefugBgIJG3zel3P4wDQYJKoZIhvcNAQELBQAw
HDEaMBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcN
MzEwMTAxMDAwMDAwWjAZMRcwFQYDVQQDEw5TbWFsbHN0ZXAgTGVhZjCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAOAUZVzOK9Apmcjfq5x9o48EQLmZm/Xb
hbWQGV093yIecsWJD3Rx/As5fo4TLdzS8Xao6dlXqzlVRNjvCmPATs6dJnVP1tm/
aqeVFwKHqh5lhHqx93FYKTPbMcpQHGfVNY8rD3/aLrTbcWiqGd9wrKUAUJO8LOS4
HhFsjcUYwulXkosHiYioZImksq1Z/cu1L3fzUuPH5WyRk8Nh+aXUuXpHGxTS1djS
p7d3OjEvK5Ob8AJQjkY3VS90KlxqQ94epPXPxp0hdYpOxbswXU0FMF4SeO1yc/QW
KeqKtPMvRwg+IXwkTHqDRyZMHn5xrvAOaV1jkp+mjDptoQpUKzB0OXUCAwEAAaOB
njCBmzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUF
BwMCMB0GA1UdDgQWBBQGnpb0nk82jVI3JImnfdHR7DYjADAfBgNVHSMEGDAWgBQd
z+3MFEz93EC48t0r750mEC8/wzAqBgNVHREEIzAhggxleGFtcGxlLnRlc3SCEW1h
aWwuZXhhbXBsZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQAKNre3khwoMoL0LoQG
UF8HVi5OxzzKCzueZQMU8fj/MXQhpPWAkXKa4vXwCDvVRBCC1j4u+xSeiqXvMVcR
n7QyxvYKJRnQ0k/x1zp8N6eed2tOFOz+gyHkoNSr/l9fQsAhqoL0FeVatqczI0co
DGg5ux5bjZwllFYw6LRIuhtZ4BxIQO4GC5pysrvjXb782v0iAowQHL3yC0x/Eyfr
ca/qovVST8zoWYf/1pQ/7Kp9do8VraB4dyr5r/zAy0GHPfia8qL864wTqGcuRnW3
1y2BOcKLgnTNCFp2ncWodmObsxom8KgUAVyW06cDx4XI1wa2FSx4G5mehBgSd/mQ
1Avy
-----END CERTIFICATE-----
subject=CN = Smallstep Leaf
issuer=CN = Smallstep Root CA
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1528 bytes and written 391 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 35B928206CF9C7C5082EC6DEAAC1265D497BA6239B6BF2478BA49354BACE349B
    Session-ID-ctx:
    Master-Key: 6FE2773F91AC84DF8AB12B45A3BE749BC7A6998D198A2A5F79996AE649A2FA14367585E728E5EB970E01A6EADE971A70
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 75 92 68 7b 7a f2 28 bb-4e 97 04 d3 2c ee 18 f9   u.h{z.(.N...,...
    0010 - 30 2c 03 05 03 77 0b e4-4d da 03 6f a4 20 ed 4c   0,...w..M..o. .L
    0020 - 89 f2 94 8b e0 45 5e d3-d4 b3 ab e2 83 7b 7f b1   .....E^......{..
    0030 - 00 43 6a 89 2a 8f 57 20-d3 15 c6 38 fb c9 bb 59   .Cj.*.W ...8...Y
    0040 - b5 06 ee a7 9a af 5b 26-cd 61 45 fe 0c 42 a2 03   ......[&.aE..B..
    0050 - 26 9f 96 85 25 1e b6 fb-3d c5 88 62 f0 9c c3 d0   &...%...=..b....
    0060 - e3 49 28 ad 9a 7a c1 df-4b 74 f5 e2 59 1c 57 a6   .I(..z..Kt..Y.W.
    0070 - 0e ff a1 ed be 0b e7 02-06 10 db 24 93 d0 6b 55   ...........$..kU
    0080 - c6 54 c0 fa d3 a6 d6 4a-76 dc ba a9 22 ba 2a d0   .T.....Jv...".*.
    0090 - 56 38 af 4c f5 28 c5 40-c0 64 6c 1c 4a 6f 47 24   V8.L.(.@.dl.JoG$
    00a0 - 07 7a a4 80 ea e0 3d 49-27 92 a5 c3 59 98 50 50   .z....=I'...Y.PP
    00b0 - f3 01 f9 ce 39 11 5b 46-9d d3 9d 27 cb 74 9c ea   ....9.[F...'.t..

    Start Time: 1757209682
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN] Dovecot (Debian) ready.

Excluding the DHE key-exchange ciphersuites then allows to negotiate a compatible ciphersuite (ECDHE-RSA-AES128-SHA256):

$ timeout 1 openssl s_client \
  -tls1_2 \
  -cipher 'ALL:!kDHE:!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

CONNECTED(00000003)
depth=1 CN = Smallstep Root CA
verify return:1
depth=0 CN = Smallstep Leaf
verify return:1
---
Certificate chain
 0 s:CN = Smallstep Leaf
   i:CN = Smallstep Root CA
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jan  1 00:00:00 2021 GMT; NotAfter: Jan  1 00:00:00 2031 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDXzCCAkegAwIBAgIRAKx4AavefugBgIJG3zel3P4wDQYJKoZIhvcNAQELBQAw
HDEaMBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcN
MzEwMTAxMDAwMDAwWjAZMRcwFQYDVQQDEw5TbWFsbHN0ZXAgTGVhZjCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAOAUZVzOK9Apmcjfq5x9o48EQLmZm/Xb
hbWQGV093yIecsWJD3Rx/As5fo4TLdzS8Xao6dlXqzlVRNjvCmPATs6dJnVP1tm/
aqeVFwKHqh5lhHqx93FYKTPbMcpQHGfVNY8rD3/aLrTbcWiqGd9wrKUAUJO8LOS4
HhFsjcUYwulXkosHiYioZImksq1Z/cu1L3fzUuPH5WyRk8Nh+aXUuXpHGxTS1djS
p7d3OjEvK5Ob8AJQjkY3VS90KlxqQ94epPXPxp0hdYpOxbswXU0FMF4SeO1yc/QW
KeqKtPMvRwg+IXwkTHqDRyZMHn5xrvAOaV1jkp+mjDptoQpUKzB0OXUCAwEAAaOB
njCBmzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUF
BwMCMB0GA1UdDgQWBBQGnpb0nk82jVI3JImnfdHR7DYjADAfBgNVHSMEGDAWgBQd
z+3MFEz93EC48t0r750mEC8/wzAqBgNVHREEIzAhggxleGFtcGxlLnRlc3SCEW1h
aWwuZXhhbXBsZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQAKNre3khwoMoL0LoQG
UF8HVi5OxzzKCzueZQMU8fj/MXQhpPWAkXKa4vXwCDvVRBCC1j4u+xSeiqXvMVcR
n7QyxvYKJRnQ0k/x1zp8N6eed2tOFOz+gyHkoNSr/l9fQsAhqoL0FeVatqczI0co
DGg5ux5bjZwllFYw6LRIuhtZ4BxIQO4GC5pysrvjXb782v0iAowQHL3yC0x/Eyfr
ca/qovVST8zoWYf/1pQ/7Kp9do8VraB4dyr5r/zAy0GHPfia8qL864wTqGcuRnW3
1y2BOcKLgnTNCFp2ncWodmObsxom8KgUAVyW06cDx4XI1wa2FSx4G5mehBgSd/mQ
1Avy
-----END CERTIFICATE-----
subject=CN = Smallstep Leaf
issuer=CN = Smallstep Root CA
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1572 bytes and written 371 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-SHA256
    Session-ID: BCF19027B9D1B92A9C25995E2A06D826DCFF35C6B1D98746AD55D50851261ED7
    Session-ID-ctx:
    Master-Key: E88430B77985A3398CD452B8B8C781F62DFE714C3B9D76BCECE188CA8D0480A2115F91581D167694120D0FCF568A5F7C
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 5b b7 e4 ac 69 34 5b 18-47 79 b2 9a 9b 01 75 58   [...i4[.Gy....uX
    0010 - ff 16 26 a8 3c 24 fc e3-2a f5 ba 23 8b fc 70 19   ..&.<$..*..#..p.
    0020 - f5 9a 74 fc 43 d5 67 82-0a 3e 20 7d 26 a1 b3 19   ..t.C.g..> }&...
    0030 - 6c 47 7a e7 08 20 66 05-b6 89 11 c7 2c e1 81 25   lGz.. f.....,..%
    0040 - 60 f1 f0 32 79 3a db ab-f1 b1 63 89 ed 13 08 3d   `..2y:....c....=
    0050 - 7c 8a aa d8 b9 a6 1f 48-19 ff d4 b3 4b f0 30 d1   |......H....K.0.
    0060 - ab 78 a2 ca 29 10 dc 05-18 74 01 9b 3f 7e 68 ce   .x..)....t..?~h.
    0070 - 16 ee 7e b4 08 8e 7e da-d9 1d d8 e9 44 42 07 cb   ..~...~.....DB..
    0080 - 6b 8a 1b f8 ee 76 97 e4-32 af 33 96 dd 14 c3 3c   k....v..2.3....<
    0090 - 8f d1 6b 68 48 c8 bc 9a-e6 6d f9 16 b8 f4 4f 11   ..khH....m....O.
    00a0 - b6 67 82 ec 5e 62 c4 49-7f dd fe a8 27 c7 b9 3b   .g..^b.I....'..;
    00b0 - ed 93 aa b3 ed 8e d9 11-9f a3 81 45 78 50 38 6c   ...........ExP8l

    Start Time: 1757210700
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN] Dovecot (Debian) ready.

Client preference (similar failure)

Adding the to the dovecot-overrides config in compose.yaml the setting ssl_prefer_server_ciphers = no will configure to not use server preference, which has the following results.

Now instead of checking the reported findings for server preference:

jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipherorder_TLSv1_2") | .finding' testssl/port_993.json

We can do this instead:

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --jsonfile-pretty port_993.json --overwrite --cipher-per-proto mail.example.test:993

$ jq --raw-output '.scanResult[0].cipherTests[] | select(.id == "supportedciphers_TLS 1_2") | .finding' testssl/port_993.json
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

$ jq --raw-output '.scanResult[0].cipherTests[] | select(.id == "supportedciphers_TLS 1_3") | .finding' testssl/port_993.json
TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256

Actually if using --preference instead of --cipher-per-proto / --each-cipher, the same results can be found (although this time server order cannot be determined and no cipherorder_* entries exist, instead there is supportedciphers_, which is a little inconsistent with the --cipher-per-proto example above that has a space in the key instead of a v):

$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "supportedciphers_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

Although... now this doesn't report the expected ECDHE-RSA-CHACHA20-POLY1305 for TLS 1.2? ๐Ÿ˜• (if server preference was enabled, then --cipher-per-proto only lists the same GCM + CHACHA20 ciphersuites without the additional CBC ones shown above)

Client preference - Investigating preference order

With the switch to disabling server preference, the client preference still seems to attempt to negotiate a DHE cipher before CHACHA20, which I've reproduced with OpenSSL.

I am familiar that on clients where CHACHA20 is more optimal over AES (such as ARM devices), it would be prioritized instead, however in this case since it is not, we encounter the same problem that server preference ran into unless Dovecot configures ssl_cipher_list to exclude the DHE ciphersuites (or is configured to provide a DH params file instead).

# This will fail due to the position of `ALL` being earlier than explicit cipher suite preference:
# - `ECDHE-RSA-CHACHA20-POLY1305:ALL` would adjust client preference to try CHACHA20 first
# - Alternatively omitting negation of `ECDHE-RSA-AES256-GCM-SHA384` would give it priority in `ALL`
timeout 1 openssl s_client \
  -tls1_2 \
  -cipher 'ALL:ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384:!ECDHE-RSA-AES128-SHA256:!ECDHE-RSA-AES256-SHA384' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

For reference, openssl ciphers will show the implicit client preference:

# Inferred client preference order of TLS 1.2 cipher suites to use for default `ALL`:
# `ECDHE-RSA-AES256-GCM-SHA384` is valid before `DHE-RSA-AES256-GCM-SHA384`,
# `ECDHE-RSA-CHACHA20-POLY1305` would then be tried if the server didn't
# misleading convey support for `DHE-RSA-AES256-GCM-SHA384`..
$ openssl ciphers -tls1_2 -s -v 'ALL'

ECDHE-ECDSA-AES256-GCM-SHA384  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256)            Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384    TLSv1.2 Kx=ECDH     Au=RSA   Enc=AESGCM(256)            Mac=AEAD
DHE-DSS-AES256-GCM-SHA384      TLSv1.2 Kx=DH       Au=DSS   Enc=AESGCM(256)            Mac=AEAD
DHE-RSA-AES256-GCM-SHA384      TLSv1.2 Kx=DH       Au=RSA   Enc=AESGCM(256)            Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305    TLSv1.2 Kx=ECDH     Au=RSA   Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305      TLSv1.2 Kx=DH       Au=RSA   Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES256-CCM8        TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESCCM8(256)           Mac=AEAD
ECDHE-ECDSA-AES256-CCM         TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESCCM(256)            Mac=AEAD
DHE-RSA-AES256-CCM8            TLSv1.2 Kx=DH       Au=RSA   Enc=AESCCM8(256)           Mac=AEAD
DHE-RSA-AES256-CCM             TLSv1.2 Kx=DH       Au=RSA   Enc=AESCCM(256)            Mac=AEAD
# ...

# Without specifying a cipherlist like ALL, the default is a bit different in order:
# This is equivalent to `DEFAULT` it seems?:
$ openssl ciphers -tls1_2 -s -v

ECDHE-ECDSA-AES256-GCM-SHA384  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256)            Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384    TLSv1.2 Kx=ECDH     Au=RSA   Enc=AESGCM(256)            Mac=AEAD
DHE-RSA-AES256-GCM-SHA384      TLSv1.2 Kx=DH       Au=RSA   Enc=AESGCM(256)            Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305    TLSv1.2 Kx=ECDH     Au=RSA   Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305      TLSv1.2 Kx=DH       Au=RSA   Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128)            Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256    TLSv1.2 Kx=ECDH     Au=RSA   Enc=AESGCM(128)            Mac=AEAD
# ...

This clarifies why CHACHA20 is not listed?

Unclear how testssl.sh is negotiating client preference order differently?

However testssl.sh was reporting results with ECDHE-RSA-AES128-SHA256 + ECDHE-RSA-AES256-SHA384 despite the missing ECDHE-RSA-CHACHA20-POLY1305 ciphersuite?

# `DEFAULT` or `ALL` makes no difference, DHE ciphersuites still come earlier in client precedence:
# Only if adding `!kDHE` will it negotiate the expected `ECDHE-RSA-AES128-SHA256` + `ECDHE-RSA-AES256-SHA384`
timeout 1 openssl s_client \
  -tls1_2 \
  -cipher '!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384:ALL' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

So testssl.sh must be querying a bit differently, it's just unclear how? A different client preference?

# This setting is the equivalent value configured in the DMS 15.1 image for `TLS_LEVEL=intermediate`
# Ref: https://github.com/docker-mailserver/docker-mailserver/blob/5f4e868c541b142c705a643315ba8828d1836660/target/scripts/helpers/ssl.sh#L145-L146

ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256

Compared to this cipher list which excludes the DHE cipher suites, correcting the Dovecot ciphersuite misconfiguration, which negotiates correctly all expected the ciphersuites for a certificate:

# Removed: DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-SHA256 DHE-RSA-AES256-SHA256

ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384

Between the Debian 12 image and Alpine 3.21, openssl ciphers appears the same for DEFAULT, but differs slightly with ALL, yet I do not see a reason either would negotiate ECDHE-RSA-AES128-SHA256 + ECDHE-RSA-AES256-SHA384 before they would ECDHE-RSA-CHACHA20-POLY1305?

Alpine 3.21: OpenSSL 3.3.4 1 Jul 2025 (Library: OpenSSL 3.3.4 1 Jul 2025)
Debian 12 (DMS v15.1): OpenSSL 3.0.17 1 Jul 2025 (Library: OpenSSL 3.0.17 1 Jul 2025)

Both fail with the last openssl s_client command above, so testssl.sh is doing something different perhaps.

Reference - Dovecot defaults

The Dovecot default cipher list for ssl_cipher_list (Dovecot 2.4.1) is (same for Dovecot 2.3 in DMS 15.1):

ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH (for ssl_server, empty for ssl_client)

NOTE: Dovecot equivalent settings differ slightly by either name or values between the 2.3.x vs 2.4.x releases (DMS 15.1 uses Dovecot 2.3.x):

The ssl_dh (2.3) / ssl_server_dh_file (2.4) setting from the above Dovecot docs links are a bit vague on the importance/relevance of the setting in relation to ssl_cipher_list. As the DHE key-exchange cipher suites don't exist for ECDSA, that certificate will avoid the negotiation mishap, but RSA certificate seems to still offer the DHE ciphersuites despite a blank/unset ssl_dh preventing their usage (so a bug on Dovecot's end), requiring explicit opt-out via ssl_cipher_list via !kDHE.


Reference - Server preference impact

The below extracts some of the terminal output and JSON file results to share observed differences.

Server Preference - Disabled

Testing server's cipher preferences
TLSv1.2 (no server order, thus listed by strength)
Has server cipher order?     no (NOT ok)
 (limited sense as client will pick)
{
  "serverPreferences": [
    {
      "id": "cipher_order-tls1_2",
      "severity": "LOW",
      "finding": "NOT a cipher order configured"
    },
    {
      "id": "cipher_order-tls1_3",
      "severity": "INFO",
      "finding": "NOT a cipher order configured"
    },
    {
      "id": "supportedciphers_TLSv1_2",
      "severity": "INFO",
      "finding": "ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256"
    },
    {
      "id": "supportedciphers_TLSv1_3",
      "severity": "INFO",
      "finding": "TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256"
    },
    {
      "id": "cipher_order",
      "severity": "LOW",
      "finding": "NOT a cipher order configured"
    }
  ]
}

Server Preference - Disabled (misconfigured server)

This is when Dovecot service has not explicitly been configured to exclude DHE ciphersuites if no DH params file was configured to support negotiating those.

Testing server's cipher preferences
TLSv1.2 (listed by strength)
Has server cipher order?     unable to determine

Missing id for cipher_order (including cipher_order-tls1_2 + cipher_order-tls1_3)

{
  "serverPreferences": [
    {
      "id": "supportedciphers_TLSv1_2",
      "severity": "INFO",
      "finding": "ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256"
    },
    {
      "id": "supportedciphers_TLSv1_3",
      "severity": "INFO",
      "finding": "TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256"
    }
  ]
}

Server Preference - Enabled

Regardless of a misconfigured Dovecot, the same id and referenced terminal output lines are present.

No supportedciphers_TLSv1_2 / supportedciphers_TLSv1_3 since order is server specific? (_even though when server preference is disabled, the values for these id appear equivalent to their cipherorder-* counterparts ๐Ÿคทโ€โ™‚๏ธ

Testing server's cipher preferences
TLSv1.2 (server order)
Has server cipher order?     yes (OK) -- TLS 1.3 and below
{
  "serverPreferences": [
    {
      "id": "cipher_order-tls1_2",
      "severity": "OK",
      "finding": "server"
    },
    {
      "id": "cipher_order-tls1_3",
      "severity": "OK",
      "finding": "server"
    },
    {
      "id": "cipherorder_TLSv1_2",
      "severity": "INFO",
      "finding": "ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES256-SHA384"
    },
    {
      "id": "cipherorder_TLSv1_3",
      "severity": "INFO",
      "finding": "TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256"
    },
    {
      "id": "prioritize_chacha_TLSv1_3",
      "severity": "INFO",
      "finding": "false"
    },
    {
      "id": "cipher_order",
      "severity": "OK",
      "finding": "server"
    }
  ]
}

Hi @polarathene , dear ,

thanks for your report.

It's quite hard for me to filter out the relevant information for this project. First of all I am only interested in what is / could be wrong with testssl.sh . Then I totally do not care what any openssl version does or does not: it's not relevant for a comparison with testssl.sh, see FAQ. FYI: Most of the time testssl.sh is right.

Therefore I normally would need the FQDN of the target to reproduce what's going on --see template for filing bugs. I know in your case it might not be as easy.

If you could help me with one and short description (see template) we can move forward.

It's quite hard for me to filter out the relevant information for this project. First of all I am only interested in what is / could be wrong with testssl.sh .

Understood, sorry about the verbosity but I wanted to be sure I had everything down first ๐Ÿ˜…

Therefore I normally would need the FQDN of the target to reproduce what's going on
I know in your case it might not be as easy.

You were provided one? The reproduction is using containers via Docker Compose compose.yaml, it sets up the FQDN / DNS for you between the two containers. Once you have the two images locally you can reproduce completely offline.

If you could help me with one and short description (see template) we can move forward.

The template was actually filled out at the issue description. I unfortunately shared a reproduction that was misconfigured and I addressed that in follow-up comments but for the most part it's the same described issue.

I will summarize my report below, but would like to place emphasis on the --each-cipher concern.

Regardless of server or client cipher list preference order, if the client or server offers a cipher suite to negotiate that it does not actually support, negotiation fails. This is expected since it turned out the service (Dovecot) was misconfigured to offer DHE cipher suites when no DH parameters were configured for TLS 1.2.

Presumably testssl.sh is not able to detect this misconfiguration, so when it enumerates a list of cipher suites, it is unable to warn that a cipher suite negotiated wasn't actually compatible and thus bailed early?

That's understandable. So I then raised the question about --each-cipher / --cipher-per-proto options instead of --server-preference, thinking that these would test each cipher via iterations of openssl -cipher "${CIPHERSUITE_TO_CHECK}" but my reported findings suggest that your logic might be negotiating by appending exclusions for each previous cipher suite that negotiated successfully?

The expectation was that this misconfigured DHE cipher suite support would not prevent identifying all compatible cipher suites from the service. testssl.sh does not appear to offer that functionality, but openssl can successfully negotiate a cipher suite directly via -cipher.

The final observation I raised was then that testssl.sh is connecting with an unknown client preference order, which does not match openssl -cipher DEFAULT or openssl -cipher ALL, since ECDHE AES-CBC cipher suites were being negotiated before the CHACHA20-POLY1305 cipher suite (which testssl.sh failed to reach as a DHE cipher suite preferred prior which would cause testssl.sh to bail early). With direct openssl commands CHACHA20-POLY1305 would be negotiated before the ECDHE AES-CBC cipher suites, hence the observation given that the FAQ makes no mention of the custom preference used.

If you could help me with one and short description (see template) we can move forward.

@drwetter This comment appears verbose, but has been structured to the satisfy the bug report template requirements.

  • Reproduction (environment + commands, with additional context)
  • Cases 1 to 3 with Actual vs Expected.
  • Supplementary section on cipher order detection observation.
  • The Docker image (3.3dev with Alpine via ghcr.io registry) is used (this doesn't output a specific commit hash via testssl -v)

Focus should be on the assumed bug (in logic or documentation) with --each-cipher. The concern is described in the bug summary bullet point that follows next (please see section "Case 3" for actual vs expected).


To summarize:

  • BUG?: testssl.sh with --each-cipher / --cipher-per-proto was expected to report all cipher suites compatible with the service. It appears to be implemented in a manner that will fail to correctly report supported cipher suites should any negotiation fail before reaching the end of the cipher list.
  • Undocumented: testssl.sh appears to have a custom undocumented client preference. This was relevant when troubleshooting with OpenSSL directly to try replicate the same failure point/cause where testssl.sh halted it's findings early (due to the misconfigured service offering DHE cipher suites that it could not support).
  • JSON UX: testssl.sh with --server-preference is presumably unable to detect/report this failure scenario (regardless of server or client preference), but there are 3 variations in the terminal output with one variant aware (misconfigured service with client preference reports on terminal only "unable to determine"), but not communicated in the JSON report (unless inferring from the omission of a cipher_order id entry).

I've provided below a revised compose.yaml.

  • This provides a full reproduction environment with TLS and FQDN (DNS) setup that can be used completely offline.
  • testssl.sh is used via it's official Docker image, you can ignore the compose.yaml and just look at the docker compose run command to see the CLI args passed to testssl for it's command.
  • The service itself (Dovecot / DMS) is not important here, it is solely about a service offering a cipher suite that cannot actually be negotiated successfully, and the unexpected (incomplete) results from testssl.sh.

Reproduction

This reproduction uses two ENV to simplify running the different configurations for actual vs expected, here are the defaults:

  • DOVECOT_CONFIG=fail (uses a misconfigured service)
  • SERVER_PREFERENCE=yes (cipher list preference during negotiation)

With Docker Compose, each actual/expected example below will correctly setup containers for testssl.sh via this command (with configurable ENV prepended):

SERVER_PREFERENCE=no DOVECOT_CONFIG=pass docker compose up -d --force-recreate

Followed by running a command like the following to the testssl container:

# NOTE: `--volume` + `--workdir` + `--user` are all for supporting access to `--jsonfile-pretty` output

docker compose run --rm \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

The CLI utility jq is then used to check the JSON report for actual vs expected results.


Use the following compose.yaml:

name: example

services:
  testssl:
    scale: 0 # This avoids running the container during `docker compose up`
    image: ghcr.io/testssl/testssl.sh:3.3dev
    # Config for `testssl.sh` to trust the locally generated cert:
    environment:
      ADDTL_CA_FILES: "/tmp/tls/ca-cert.pem"
    configs:
      - source: tls-ca-cert
        target: /tmp/tls/ca-cert.pem

  # Service to test against:
  dms:
    image: mailserver/docker-mailserver:15.1.0
    hostname: mail.example.test
    environment:
      - SSL_TYPE=manual
      - SSL_KEY_PATH=/tmp/tls/key.pem
      - SSL_CERT_PATH=/tmp/tls/cert.pem
      - TLS_LEVEL=intermediate
    configs:
      # Extra config bundled into the `compose.yaml` to ease testing:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf
      - source: tls-cert
        target: /tmp/tls/cert.pem
      - source: tls-key
        target: /tmp/tls/key.pem
      # Required to reproduce the observed bug by removing DHE support (while retaining DHE in server order):
      - source: dovecot-overrides-${DOVECOT_CONFIG:-fail}
        target: /tmp/docker-mailserver/dovecot.cf
      # Optional - Allows clients in the container to verify cert trust with the CA that signed it:
      - source: tls-ca-cert
        target: /tmp/tls/ca-cert.pem

configs:
  # Pre-provisioned account for local reproduction use:
  # NOTE: `$` is escaped by repeating it to avoid the Docker Compose ENV interpolation feature
  # Password is `secret`
  dms-accounts:
    content: |
      john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.

  # Example RSA cert files for testing locally (provisioned for `mail.example.test`):
  tls-ca-cert:
    content: |
      -----BEGIN CERTIFICATE-----
      MIIDCDCCAfCgAwIBAgIRANPCP+3+4OKj3S2gIsfQcoIwDQYJKoZIhvcNAQELBQAw
      HDEaMBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcN
      MzEwMTAxMDAwMDAwWjAcMRowGAYDVQQDExFTbWFsbHN0ZXAgUm9vdCBDQTCCASIw
      DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKuYF7paXiAvIK8IIE6rnNMSuS94
      z8+1md2a6DnijBiwG/DcB2e1wxBab/kK+XNsPWI26FLUxqc9zGZ0Nn7s7DOT6C52
      43m4vJUrTthsFdLzW9SFCbMFCikYhbOSqfj06oVHRstkAI+WgyX6QrnrSV+h0imk
      gZlwjk+oKpkE6rS8zCGgXZMqi7/06qiTwXCVStMnEnX9FuNTbMjgz6bVQH6FOE9e
      8QbykSUjWvE+x6Rk5gUpbFGV26Bb62zH6zF9FY6ECsdsJJlALJ7e5w6MTfSL23u8
      cRcBhtBaQuHkFg9G6JFigQwbm0DQf7o7crdMJY/paWbCuRhxXx3YGZFOU1sCAwEA
      AaNFMEMwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQIMAYBAf8CAQEwHQYDVR0O
      BBYEFB3P7cwUTP3cQLjy3SvvnSYQLz/DMA0GCSqGSIb3DQEBCwUAA4IBAQBcSu9x
      eGVGMjfxOmX1S+MyMJNPWKJuBlW5cX80h7mBIVIKnamJjMBDGu8WJFnaFdch5uwr
      KIlXbDmFJjxBb3id/6W3BC09Ze6fjtY+rPRPkCi5L5CbLs6KU1V2X/xoMhg15upH
      e29Giz2gK/Zd9/ewxa+sNRx2mk9fhIzgLA55q34OkhEdi0dJFlyzNHw+/6eYICPd
      hPv+8z3dreuyXqKBO+M/QqLXHqonvZWbJzeWCqetZn/h9WFM2WXMzRTR/GjaUIQP
      /ZPLXm5P4YMtAEbG+PfHuzC0bwUmj5FTGJ7I6v9Y32KS+P94KlsmHOJICqSwLQWo
      6MbojAT5BXYzHzHq
      -----END CERTIFICATE-----

  tls-key:
    content: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEpAIBAAKCAQEA4BRlXM4r0CmZyN+rnH2jjwRAuZmb9duFtZAZXT3fIh5yxYkP
      dHH8Czl+jhMt3NLxdqjp2VerOVVE2O8KY8BOzp0mdU/W2b9qp5UXAoeqHmWEerH3
      cVgpM9sxylAcZ9U1jysPf9outNtxaKoZ33CspQBQk7ws5LgeEWyNxRjC6VeSiweJ
      iKhkiaSyrVn9y7Uvd/NS48flbJGTw2H5pdS5ekcbFNLV2NKnt3c6MS8rk5vwAlCO
      RjdVL3QqXGpD3h6k9c/GnSF1ik7FuzBdTQUwXhJ47XJz9BYp6oq08y9HCD4hfCRM
      eoNHJkwefnGu8A5pXWOSn6aMOm2hClQrMHQ5dQIDAQABAoIBAH6Yx1ORX4txlWK5
      i1kUWm2Yd4DkWgqjBX24dbwKEqBSF4Gml8awBzfIOcnG6ChUPPtPHx8duqzfkdAF
      7RbCNUPh4TJx8u5+iKE5SBCz2Nbnf9tZ5HRy3IRhmFW2wPWgrWu/ZjhTagPf3sjF
      IWztWXy3Gs78h1iI9OPfMpFiFeyB8LEE1w4nb/iSSUvFue/VZ0aDS5eOmUFVzjxy
      xNfHZ33QnmRYbsJ58oF2Fr+3fTmltM406y+Tg3+Dao+Mpf41NKq+1r58PnKpUjfH
      cLO5k4/Mqbnk4xi/ftzApIew1A6ClO9xwA3+oYE/S0LQ8JHEVajG1aMw8PEXWbOJ
      wzcXwEECgYEA7NZQDQHjCCzdmECITSr3I3tuXbmnv29RS4sAepn9fS4OVosnntrc
      YnRGriuFIZpyrr/3R5DwTgO61VeBAqvJDYtVjj6zyT9f87fdm8gJeDW7T0G24vt6
      0H4KG8ws8+I/FjMj7wiC4yN4/Hcyd94squh+/9NjdTvwfvS6frwnQR0CgYEA8jXV
      SRfBYJ8U8VBMZxfOmvEtJfxaahtnJGBBEwDcCfgB+rIjVQGw0aZONPKilD8jinAt
      UiBwyBDkHaUL14s6H23+QS6am6Mr9lTz+YPqpYWG0VjGBv8kuHZ50EhlBJFadzMb
      VVR5R5FDP3ChMQy7Q6e8RoIK5DQD86vKjQRegjkCgYAlERGsR3xR3ju8RXVPpobR
      bdMDJjhj1LdDfHjRt2IeAmRKFTNZQGW3nv0k6zjF3pdOVEsOT1fczeai1zQgx+QK
      k6ELRzL6L0oEKeWsKO2ae8ZaDC3kbnl1QhSw7w6mCOXYwp5AHfPmOroHwVwLuKED
      Cqo9vcbWJVBpfkHl7eqy3QKBgQCHOZDrbuzSsd4yX79YK0146b9oHryn0sbB409R
      ecBffGw2d7AMLJZ4Zd3x56jnFV0VVE2pNV1iBTQmbNfwrdV0aKdz4r4EuJO5wnI3
      0vN1F9hOFr7wdxAcQGD/7PshErmsJQdUm4Xec/ZUe+Ayj0YZnpMZ1k6YW4X9S+MY
      2eCd2QKBgQDQRh7HHS3n7PAwwtUDKR9oEFWE0dLOr+RWSeFsXz10gxUbQuHvLXU4
      nOCGA+Add6995PAs9xcnA/Ewju/l3YBgiQPvmLqEnpSZUiRoKQcTE+sAd9FCNB/4
      lORhokK5KlIkxUQISAZ65p0awQe0yyfOD1VJmcvALF1FRwWyz5M1TA==
      -----END RSA PRIVATE KEY-----

  tls-cert:
    content: |
      -----BEGIN CERTIFICATE-----
      MIIDXzCCAkegAwIBAgIRAKx4AavefugBgIJG3zel3P4wDQYJKoZIhvcNAQELBQAw
      HDEaMBgGA1UEAxMRU21hbGxzdGVwIFJvb3QgQ0EwHhcNMjEwMTAxMDAwMDAwWhcN
      MzEwMTAxMDAwMDAwWjAZMRcwFQYDVQQDEw5TbWFsbHN0ZXAgTGVhZjCCASIwDQYJ
      KoZIhvcNAQEBBQADggEPADCCAQoCggEBAOAUZVzOK9Apmcjfq5x9o48EQLmZm/Xb
      hbWQGV093yIecsWJD3Rx/As5fo4TLdzS8Xao6dlXqzlVRNjvCmPATs6dJnVP1tm/
      aqeVFwKHqh5lhHqx93FYKTPbMcpQHGfVNY8rD3/aLrTbcWiqGd9wrKUAUJO8LOS4
      HhFsjcUYwulXkosHiYioZImksq1Z/cu1L3fzUuPH5WyRk8Nh+aXUuXpHGxTS1djS
      p7d3OjEvK5Ob8AJQjkY3VS90KlxqQ94epPXPxp0hdYpOxbswXU0FMF4SeO1yc/QW
      KeqKtPMvRwg+IXwkTHqDRyZMHn5xrvAOaV1jkp+mjDptoQpUKzB0OXUCAwEAAaOB
      njCBmzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUF
      BwMCMB0GA1UdDgQWBBQGnpb0nk82jVI3JImnfdHR7DYjADAfBgNVHSMEGDAWgBQd
      z+3MFEz93EC48t0r750mEC8/wzAqBgNVHREEIzAhggxleGFtcGxlLnRlc3SCEW1h
      aWwuZXhhbXBsZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQAKNre3khwoMoL0LoQG
      UF8HVi5OxzzKCzueZQMU8fj/MXQhpPWAkXKa4vXwCDvVRBCC1j4u+xSeiqXvMVcR
      n7QyxvYKJRnQ0k/x1zp8N6eed2tOFOz+gyHkoNSr/l9fQsAhqoL0FeVatqczI0co
      DGg5ux5bjZwllFYw6LRIuhtZ4BxIQO4GC5pysrvjXb782v0iAowQHL3yC0x/Eyfr
      ca/qovVST8zoWYf/1pQ/7Kp9do8VraB4dyr5r/zAy0GHPfia8qL864wTqGcuRnW3
      1y2BOcKLgnTNCFp2ncWodmObsxom8KgUAVyW06cDx4XI1wa2FSx4G5mehBgSd/mQ
      1Avy
      -----END CERTIFICATE-----

  # This misconfigures the Dovecot service which is offering DHE cipher suites
  # but without `ssl_dh` setting DH parameters, those cipher suites will fail on negotiation
  # As soon as the client (OpenSSL/testssl.sh) fail the agreed cipher suite via preference,
  # they will stop attempting to negotiate any further cipher suites (expected).
  #
  # BUG?: `testssl.sh` should continue with `--each-cipher` however?
  dovecot-overrides-fail:
    content: |
      # Unset this setting to no longer support negotiating DHE cipher suites
      ssl_dh =

      # Use client preference:
      ssl_prefer_server_ciphers = ${SERVER_PREFERENCE:-yes}

  # Configures `ssl_cipher_list` without the DHE cipher suites (in addition to unsetting `ssl_dh`),
  # Dovecot is no longer misconfigured to offer DHE cipher suites in it's cipher list
  dovecot-overrides-pass:
    content: |
      # Unset this setting to no longer support negotiating DHE cipher suites
      ssl_dh =

      # Use client preference:
      ssl_prefer_server_ciphers = ${SERVER_PREFERENCE:-yes}

      # This setting is the equivalent value configured in the DMS image, except the DHE cipher suites have been removed.
      # Ref: https://github.com/docker-mailserver/docker-mailserver/blob/5f4e868c541b142c705a643315ba8828d1836660/target/scripts/helpers/ssl.sh#L145-L146
      ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384

Case 1 - Server Preference (non-issue)

Actual

Server preference with misconfigured (no DH parameters set) DHE cipher suites offered by Dovecot:

$ DOVECOT_CONFIG=fail docker compose up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Actual result:
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipherorder_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384

NOTE: This would be the same outcome from negotiating with an openssl command directly without testssl.sh involved, due to the server preference for cipher suite selection.

testssl.sh presumably cannot detect this mishap, so I do not deem this case an issue.

Expected (includes ECDHE-ECDSA-AES128-SHA256 + ECDHE-ECDSA-AES256-SHA384)

Server preference without DHE cipher suites offered by Dovecot:

$ DOVECOT_CONFIG=pass docker compose up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Expected result (includes `ECDHE-ECDSA-AES128-SHA256` + `ECDHE-ECDSA-AES256-SHA384`):
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipherorder_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-ECDSA-CHACHA20-POLY1305 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES256-SHA384

Case 2 - Client Preference (non-issue)

NOTE: Undocumented client preference from testssl.sh.

  • I was unable to reproduce the failure via openssl directly with DEFAULT and ALL cipher lists, which produced a different preference order.
  • I documented this difference in this earlier comment.

Actual

Client preference with misconfigured (no DH parameters set) DHE cipher suites offered by Dovecot:

$ DOVECOT_CONFIG=fail SERVER_PREFERENCE=no docker compose up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Actual result:
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "supportedciphers_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

NOTE: This would be a similar (different client preference) outcome from negotiating with an openssl command directly without testssl.sh involved.

testssl.sh presumably cannot detect this mishap, so I do not deem this case an issue.

Expected (includes ECDHE-RSA-CHACHA20-POLY1305)

Server preference without DHE cipher suites offered by Dovecot:

$ DOVECOT_CONFIG=pass SERVER_PREFERENCE=no docker compose up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Expected result (includes `ECDHE-RSA-CHACHA20-POLY1305`):
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "supportedciphers_TLSv1_2") | .finding' testssl/port_993.json
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

Case 3 - --each-cipher / --cipher-per-proto

The expectation here is for avoiding the early bail outcome that cases 1 and 2 hit. --each-cipher implies each cipher suite will be tried directly, ignoring client/server preference by using openssl -cipher "${CIPHERSUITE_HERE}" but this does not seem to be the case as documented in this earlier comment.

It seems that testssl.sh instead negotiates via preference, then appends the successfully negotiated cipher suite to the -cipher list as an exclusion, repeating this process until it fails to negotiate successfully.

This appears to be a bug? A client that does not offer DHE cipher suites for example (-cipher '!kDHE' would negotiate a cipher suite successfully, despite the misconfigured Dovecot cipher list.

The expectation is that testssl.sh would properly communicate the available cipher suites offered / negotiated. Doing so would more easily allow comparing the JSON reported finding against the software (Dovecot) cipher list config to identify the cipher suites that failed to negotiate due to misconfiguration.

NOTE: While --each-cipher could be used for this case, --cipher-per-proto was preferred as that provides a TLS 1.2 cipher list in the findings output, which simplifies actual vs expected.

For --each-cipher (or --cipher-per-proto) you could check actual vs expected results like so:

# Outputs `true` when successful (each expected cipher suite was matched to a finding in the JSON report):
$ jq --raw-output \
  --argjson expected '["ECDHE-RSA-CHACHA20-POLY1305", "ECDHE-RSA-AES256-GCM-SHA384", "ECDHE-RSA-AES128-GCM-SHA256", "ECDHE-RSA-AES256-SHA384", "ECDHE-RSA-AES128-SHA256"]' \
  '[.scanResult[0].cipherTests[].finding | select(contains($expected[]) )] as $actual | all($expected; inside($actual))' \
  testssl/port_993.json

Actual

  • Client or Server preference can be used, no difference.
  • With misconfigured (no DH parameters set) DHE cipher suites offered by Dovecot
$ DOVECOT_CONFIG=fail SERVER_PREFERENCE=no docker compose up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --cipher-per-proto --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Actual result:
$ jq --raw-output '.scanResult[0].cipherTests[] | select(.id == "supportedciphers_TLS 1_2") | .finding' testssl/port_993.json
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

Cipher suite preference (server or client) should not be influencing explicit cipher suite choice? Incorrectly documented

Expected (includes ECDHE-RSA-CHACHA20-POLY1305)

Server preference without DHE cipher suites offered by Dovecot:

$ DOVECOT_CONFIG=pass SERVER_PREFERENCE=no docker compose up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --cipher-per-proto --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# Expected result (includes `ECDHE-RSA-CHACHA20-POLY1305`):
$ jq --raw-output '.scanResult[0].cipherTests[] | select(.id == "supportedciphers_TLS 1_2") | .finding' testssl/port_993.json
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

Cipher Order detection

These were documented observations from this earlier comment.

Server

# NOTE: Regardless of DOVECOT_CONFIG using `pass` or `fail`, same outcome with `SERVER_PREFERENCE=yes`:
$ DOVECOT_CONFIG=pass SERVER_PREFERENCE=yes docker compose -f compose.report.yaml up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# OK - Logs that server preference is not in use:
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipher_order-tls1_2")' testssl/port_993.json

{
  "id": "cipher_order-tls1_2",
  "severity": "OK",
  "finding": "server"
}

Client preference (detected)

$ DOVECOT_CONFIG=pass SERVER_PREFERENCE=no docker compose -f compose.report.yaml up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# OK - Logs that server preference is not in use:
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipher_order-tls1_2")' testssl/port_993.json

{
  "id": "cipher_order-tls1_2",
  "severity": "LOW",
  "finding": "NOT a cipher order configured"
}

Client preference (uncertain)

According to terminal output from testssl.sh it was unable to determine server preference.

$ DOVECOT_CONFIG=fail SERVER_PREFERENCE=no docker compose -f compose.report.yaml up -d --force-recreate

$ docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

# NOT OK? - Entry missing (terminal output from `testssl.sh` command logged "Has server cipher order?     unable to determine"):
$ jq --raw-output '.scanResult[0].serverPreferences[] | select(.id == "cipher_order-tls1_2")' testssl/port_993.json

If it makes it easier, I've adapted the previous comment to the actual template.


Before you open an issue please check which version you are running and whether it is the latest in stable / dev branch

GHCR.io 3.3dev image (Sep 6 2025)

Before you open an issue please whether this is a known problem by searching the issues

I couldn't find anything

Command line / docker command to reproduce

testssl --quiet --each-cipher mail.example.test:993

NOTE: The FQDN mail.example.test represents a service that offers DHE cipher suites for TLS 1.2 without DH parameters configured. As a result negotiating such cipher suites will fail.

  • This is a misconfiguration and is easily fixed, the bug report is regarding how --each-cipher handled this scenario.
  • This is not running on a live server, it is fully reproducible via Docker Compose with the compose.yaml as demonstrated in the previous comment: #2883 (comment)

Expected behavior

I expected all cipher suites supported by the service to be reported, as --each-cipher implies it would.

Instead this option presumably does not try each cipher suite explicitly, but does so implicitly during negotiation?

# Actual:
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

# Expected (includes `ECDHE-RSA-CHACHA20-POLY1305`):
ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-CHACHA20-POLY1305 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256

Your system (please complete the following information):

  • N/A (using the GHCR.io image)

Additional context

See the previous comment for additional context: #2883 (comment)

It would be easier to have the ability to check this myself rather than digging in the code and continue to figure out what is wrong.

Any idea?

It would be easier to have the ability to check this myself

I'm not sure what you're asking for here?

I provided a full reproduction that you can run locally with Docker Compose (or the equivalent separate docker run commands but compose.yaml is much simpler and self-contained).

I also provided the terse report with reference links as an alternative that would be easier to digest the gist of the issue.


To clarify, you can reproduce this with any service/server offering a cipher suite for the client to negotiate with, where either the client or server are lying about actually supporting the cipher suite to negotiate (DHE cipher suites on TLS 1.2 without the server providing DH parameters in this case).

Since negotiation will fail on this DHE cipher suite due to misconfiguration of the service (Dovecot), any further negotiations by testssl.sh halt (which is likely due to how testssl.sh has implemented --each-cipher / --cipher-per-proto?)

It can be a client or server preference controlled incompatible cipher suite, so long as both offer it to negotiate.


Reproduce

Using the Docker image we're going to run this basic command (but additionally with JSON file output):

testssl --quiet --each-cipher mail.example.test:993

--each-cipher expected vs actual results and the commands are shared under Case 3 section here.

1. Start the servive (with misconfiguration) to test against

Start the containers from the single compose.yaml file (copy/paste from the top of this comments "reproduction" section):

DOVECOT_CONFIG=fail SERVER_PREFERENCE=no docker compose up -d --force-recreate

2. Run testssl.sh against the misconfigured service

Run testssl.sh (this could just be standard docker run but the compose.yaml provides some conveniences like sorting out DNS for mail.example.test):

docker compose run --rm -it \
  --volume ./testssl:/output \
  --workdir /output \
  --user "$(id -u):$(id -g)" \
  testssl --quiet --server-preference --jsonfile-pretty port_993.json --overwrite mail.example.test:993

3. Check reported cipher suites

With the JSON results output, observe only 4 of the 5 expected cipher suites was reported:

jq --raw-output '.scanResult[0].cipherTests[] | select(.id == "supportedciphers_TLS 1_2") | .finding' testssl/port_993.json

If you do not consider this report that much of a concern to spend time tackling, that is perfectly ok and you can close this issue or leave it open without resolving.

I just wanted to report it for awareness as the documented --each-cipher feature appears to imply it would test each cipher, not break because a single cipher suite failed to negotiate.

Hello @polarathene,

I can't reproduce the issue you are trying to raise, as I am not in a position to run some Docker image. However, if I understand correctly, the issue is a result of a problem with the server, and you expect testssl.sh to work around the server's bug.

As you note, testssl.sh does not test each of the 372 cipher suites in etc/cipher-mapping.txt one at a time. However, it shouldn't need to. If a client sends a ClientHello listing 50 cipher suites, then (assuming no other issues) the server should return a ServerHello with one of those 50 cipher suites, unless it does not accept any of them. So, if testssl.sh sends a ClientHello with 50 cipher suites, and the connection does not succeed, then testssl.sh considers that it has tested all 50 of those cipher suites and has determined that the server does not support any of them.

I don't know what you mean by "lying about actually supporting the cipher suite to negotiate." However, my best guess is that the internal processing of the server is selecting a DHE cipher suite, and then fails when it discovers that it doesn't have any DHE parameters. (I don't know what the server's response to the client looks like in this case.) This is a bug in the server. The server should not select a cipher suite that it cannot support, and then fail because it doesn't support that cipher suite. In this case, when the server is processing the configuration file, it should either not include the DHE cipher suites in its internal list of supported cipher suites, or it should flag an error due to an inconsistency in the configuration information it is provided.

There are some places in which testssl.sh has made an effort to work around server bugs. It is not clear at the moment, however, how a bug like this could be worked around without resulting an an unacceptable slowdown in testssl.sh's performance.

TL;DR: @dcooper16

  • I agree, this is a bug with the server software. However this issue is about the docs + UX with testssl.sh and if it could better communicate/identify the problem to the user in a report (I've noted an observation regarding client preference detection differing when this early failure occurs).
  • In the "Extra Context" section below, I've expanded on --each-cipher --show-all output being a bit misleading, along with the options docs that led to the confusion of my expectation from using it. I also shared direct openssl -cipher commands with output, but I'm not sure if testssl.sh is invoking the same for it's equivalent logic.
  • As this is a niche concern, I don't mind it remaining unresolved. I have no further need for that, but it might be helpful to future users investigating why testssl.sh reports only a partial list of supported cipher suites. A full scan of compatible cipher suites to report could assist in troubleshooting and be an opt-in flag (or option with openssl -cipher value to pass, such as DEFAULT:!kDHE), avoiding the performance concern you raised.

There's no expectation for a workaround here. I don't think this issue should be hidden from a user.

  • It was more about discovering why there was a mismatch in cipher suite compatibility reported (as my automated tests that detected the change, while direct openssl -cipher commands confirmed compatibility with cipher suites that testssl.sh --show-each reported as not a/v, which should have only been the DHE cipher suites).
  • The current report from testssl.sh is fine as a default, I just found --each-cipher --show-all to be misleading while troubleshooting why testssl.sh failed to report the remaining non-DHE cipher suites in the server where I only disabled DH. Bailing on first failed negotiation as a default is understandable in other contexts, but the docs for --each-cipher should clarify the caveat/expectation?

Original response

the issue is a result of a problem with the server, and you expect testssl.sh to work around the server's bug.

It would be client or server. So long as one is communicating support for a cipher suite it can't actually negotiate.

In this case it is the server, where the service offered DHE suites by default, but if it was not provided DH params to use, negotiation would fail. I agree that it's a bug in that the software (Dovecot) should not offer DHE suites when it hasn't had DH parameters provided in config.


So, if testssl.sh sends a ClientHello with 50 cipher suites, and the connection does not succeed, then testssl.sh considers that it has tested all 50 of those cipher suites and has determined that the server does not support any of them.

Ah ok, so I am not too familiar with this process. I assume this is referring to the use of openssl -cipher '<list of 50 cipher suites here>'?

  • I understand the expectation is that the connection to the server will negotiate a cipher suite that is agreed as compatible between both (with either client or server preference for the priority), select that and continue.
  • What I don't understand is how the remaining cipher suites are being tested after that point. Similar to reluctance of others to reproduce via my copy/paste Docker Compose example, I'm reluctant to sift through a rather large single shell script file ๐Ÿ˜…

The docs for --each-cipher do state this however:

-e, --each-cipher checks each of the (currently configured) 370 ciphers via openssl

So perhaps that could be worded better, as the failure scenario I encountered in this issue isn't mentioned that it bails early, instead of the implied full scan (I'm not quite sure how this option differs from the --server-preference option in that sense).


This is a bug in the server. The server should not select a cipher suite that it cannot support, and then fail because it doesn't support that cipher suite. In this case, when the server is processing the configuration file, it should either not include the DHE cipher suites in its internal list of supported cipher suites, or it should flag an error due to an inconsistency in the configuration information it is provided.

Agreed.


There are some places in which testssl.sh has made an effort to work around server bugs. It is not clear at the moment, however, how a bug like this could be worked around without resulting an an unacceptable slowdown in testssl.sh's performance.

I don't know how much information you have from the server. If the cipher suites it offers is not known during this process, then I assume you are reducing the cipher suite list offered from the client, by either dropping items from a -cipher list, or appending exclusions of successful negotiated cipher suites.

So I understand that this may equate to not being able to know if you've iterated through the full support offered by the server when this bug is hit. And that testssl.sh cannot communicate the misconfiguration issue to the user via the report, as a result of not knowing there was more cipher suites left to try.

An additional option to testssl.sh could allow for an OpenSSL cipherlist to be provided (eg: -cipher 'DEFAULT:!kDHE' would cover the default cipher list but exclude DHE key-exchange cipher suites), and then with each iteration append the negotiated cipher suite as an exclusion? However if you lack the information of the cipher suite used during a failed negotiation, this won't be any better.

That leaves the only other workaround as supporting a flag to explicitly iterate through cipher suites individually, which yes would be unacceptable as a default (hence the opt-in flag), and could be useful for troubleshooting in a situation like this where it wasn't apparent to me where the problem was coming from (it didn't help that openssl used directly was providing a different client preference than whatever testssl.sh was setting, as this resulted in the failure point shifting so different results of successful negotiations until that point were reported).


Extra context

This might be helpful at communicating the concern further.

My expectation with testssl.sh was that it could at least identify the available cipher suites. While one client might fail on the DHE issue (and that failure is useful), I would think it appropriate to be able to ask testssl.sh to report which cipher suites are valid, such that if there was a client that didn't offer DHE cipher suites to negotiate with, that testssl.sh would still list the remaining cipher suites that were compatible for that client, rather than the impression that the server didn't support the expected cipher suites for some reason (which is partly dependent upon the client).

I had noted in an earlier comment (and separate issue filed), that testssl.sh does have the ability to distinguish between client preference as unable to determine (DHE misconfigured server) and NOT a cipher order configured (client preference detected). So perhaps testssl.sh has some relevant check applicable to detecting this early bailing mishap? ๐Ÿ˜…

openssl direct errors with custom -cipher

I did document this in an earlier comment, but I'll be a bit more thorough here.

In my case I could exclude the non-DHE cipher suites supported to get a failed negotiation like this, which would communicate clearly that there was no more cipher suites in common between client and server that could be negotiated successfully (as these are the only ones the server offered that weren't DHE).

$ timeout 1 openssl s_client \
  -tls1_2 \
  -cipher '!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384:ALL' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

CONNECTED(00000003)
40A718AC5B790000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:../ssl/record/rec_layer_s3.c:322:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 303 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1758511628
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no

The same would happen if ALL was the first item of the -cipher list, as it would if I used DEFAULT instead of ALL. However DEFAULT at the end of the -cipher list value did produce a different error no cipher match:

# `DEFAULT` breaks if items were excluded prior? `ALL` is unaffected regardless of before/after placement.
# NOTE:
# - Replacing `DEFAULT` with `ALL` here would result in same failure as above instead
# - Besides the DHE cipher suites there should still be `ECDHE-RSA-AES256-SHA384` + `ECDHE-RSA-AES128-SHA256` available
$ timeout 1 openssl s_client \
  -tls1_2 \
  -cipher '!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384:DEFAULT' \
  -connect mail.example.test:993 \
  -CAfile /tmp/tls/ca-cert.pem

Call to SSL_CONF_cmd(-cipher, !ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384:DEFAULT) failed
4097C32B517D0000:error:0A0000B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match:../ssl/ssl_lib.c:2779:

I'm not quite sure how testssl.sh is connecting, so that may not be helpful.

When I allow the ECDHE-RSA-CHACHA20-POLY1305 cipher suite, it will connect successfully if I have ALL/DEFAULT at the end of the list, as expected. However when ALL/DEFAULT is at the front of the list, the DHE cipher suite has priority and results in the same failure example shown for ALL above.

# DHE would be attempted with the `unexpected eof while reading` error for this `-cipher` option:
-cipher 'DEFAULT:ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384'

This is the same failure scenario as well when no cipher suites are available to negotiate:

# Now with `!kDHE:!ECDHE-RSA-AES256-SHA384:!ECDHE-RSA-AES128-SHA256` appended:
-cipher 'DEFAULT:!ECDHE-RSA-CHACHA20-POLY1305:!ECDHE-RSA-AES128-GCM-SHA256:!ECDHE-RSA-AES256-GCM-SHA384:!kDHE:!ECDHE-RSA-AES256-SHA384:!ECDHE-RSA-AES128-SHA256'

testssl.sh with --each-cipher --show-each is misleading with not a/v

$ DOVECOT_CONFIG=pass SERVER_PREFERENCE=no docker compose up -d --force-recreate

$ docker compose run --rm -it \
  testssl --quiet --each-cipher --show-each mail.example.test:993 \
  | grep available

 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384                             available
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256                       available
 xc030   ECDHE-RSA-AES256-GCM-SHA384       ECDH 521   AESGCM      256      TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384              available
 xc028   ECDHE-RSA-AES256-SHA384           ECDH 521   AES         256      TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384              available
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH 521   ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256        available
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128      TLS_AES_128_GCM_SHA256                             available
 xc02f   ECDHE-RSA-AES128-GCM-SHA256       ECDH 521   AESGCM      128      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256              available
 xc027   ECDHE-RSA-AES128-SHA256           ECDH 521   AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256              available

When using DOVECOT_CONFIG=fail instead, then testssl.sh results output will exclude the ECDHE-RSA-CHACHA20-POLY1305 cipher suite, since a DHE cipher suite was attempted prior.

However the terminal output withouot grep filtering looks like this (implying each cipher was tested):

 Testing 372 ciphers via OpenSSL plus sockets against the server, ordered by encryption strength

Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits     Cipher Suite Name (IANA/RFC)
-----------------------------------------------------------------------------------------------------------------------------
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384                             available
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256                       available
 xcc14   ECDHE-ECDSA-CHACHA20-POLY1305-OLD ECDH       ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256_OLD  not a/v

# ...

testssl.sh is apparently unable to distinguish between a thorough test of the cipher suites vs early bail? not a/v is technically untested.

Since we know that even with the misconfiguration, testing for ECDHE-RSA-CHACHA20-POLY1305 directly would be successful to negotiate:

$ DOVECOT_CONFIG=fail SERVER_PREFERENCE=no docker compose up -d --force-recreate

$ docker compose run --rm -it testssl --quiet --each-cipher --show-each mail.example.test:993 | grep 'ECDHE-RSA-CHACHA20-POLY1305'

 xcc13   ECDHE-RSA-CHACHA20-POLY1305-OLD   ECDH       ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256_OLD    not a/v
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH       ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256        not a/v

not a/v does not equate to the cipher suite being tested, but rather that a failure to negotiate a cipher suite (or the lack of anymore offered) is where testssl.sh deems any further cipher suites as not a/v? (As opposed to testing a cipher suite, failing to negotiate it as not a/v)

That's an important distinction in this case, since when troubleshooting this discrepancy I found ECDHE-RSA-CHACHA20-POLY1305 was available (provided the client didn't offer DHE cipher suites ahead of it in it's client preference). Likewise with server preference, where the not a/v cipher suites differ due to server cipher list order vs client cipher list.

Either the reported results or documentation could better communicate this (presumably niche) scenario. But it's up to you if testssl.sh is meant to be useful in identifying such an issue and why it's reported results differed from what was expected of the service tested.

Hello @polarathene,

There seems to be a lot of confusion here, so I'll try to explain a few things:

testssl.sh does not just consist of a bunch of calls to openssl s_client ... So, you can't fully understand what testssl.sh is doing to trying some openssl commands. There is no version of OpenSSL that supports all of the TLS cipher suites that have been defined (https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4), or all of the elliptic curves, signature algorithms, etc. So, testssl.sh has its own (partial) implementation of the TLS handshake.

I doesn't seem there should be a need for the documentation to specifically warn that testssl.sh doesn't always do things exactly as OpenSSL does. There are many other clients (e.g., Chrome, Edge, Firefox), and they are all different from each other.

the issue is a result of a problem with the server, and you expect testssl.sh to work around the server's bug.

It would be client or server. So long as one is communicating support for a cipher suite it can't actually negotiate.

In this case, testssl.sh is the client, so the issue isn't with the client.

So perhaps that could be worded better, as the failure scenario I encountered in this issue isn't mentioned that it bails early, instead of the implied full scan (I'm not quite sure how this option differs from the --server-preference option in that sense).

testssl.sh doesn't "bail early." If testssl.sh sends the server a list of 50 cipher suites that it supports. If the server supports any one of these 50 cipher suites, then it is supposed to select one of them and continue. If the negotiation fails, then (in a non-buggy server) that means the server is indicating that it doesn't support any of the 50 cipher suites offered by the client.

I don't know how much information you have from the server. If the cipher suites it offers is not known during this process, then I assume you are reducing the cipher suite list offered from the client, by either dropping items from a -cipher list, or appending exclusions of successful negotiated cipher suites.

No. There is no concept of the server offer a cipher suite that "is not known during this process." The server can only choose from the cipher suites offered by the client. Since testssl.sh is the client, it knows all of the cipher suites that it offers.

The way testssl.sh works is that it sends a message to the server offering a large number (e.g., 100) of cipher suites. If the server chooses one of them, then testssl.sh knows the server supports that cipher suite, so it then sends a message offering the remaining (e.g., 99) cipher suites. It keeps doing this until the server indicates that it doesn't support any of the cipher suites offered. So, a failed negotiation means that testing is complete, not that testssl.sh is bailing early.

I had noted in an earlier comment (and #2884), that testssl.sh does have the ability to distinguish between client preference as unable to determine (DHE misconfigured server) and NOT a cipher order configured (client preference detected). So perhaps testssl.sh has some relevant check applicable to detecting this early bailing mishap? ๐Ÿ˜…

Without more information, I wouldn't be able to say why testssl.sh was unable to determine whether the server had its own cipher preference order, but it doesn't mean that testssl.sh knew that the server had a DHE misconfiguration issue.

Perhaps if we had more information about how the server was behaving we could develop a test for this particular issue. However, I don't think it would make sense to the code in general to try to work around this server bug. More appropriate would be to add a test to the --grease testing (which is not run by default) that would try to detect some specific incorrect behavior when DHE cipher suites are offered.

Summary

Actionable steps:

  • Decide if testssl.sh should support direct cipher suite compatibility checks (single cipher suite per connection instead of a full list) as a potentially useful troubleshooting opt-in flag (potentially with a way to minimize / filter the cipher list to scan, such as supported by OpenSSL with openssl ciphers -tls1_2 -s -v 'DEFAULT:!kDHE').
  • Decide if documentation for --each-cipher or similar should clarify that results may be incomplete if client and server agree to negotiate a cipher suite but the negotiation fails, which prevents detection of further compatible cipher suites.

I think that's all there is for maintainers to decide for resolving this issue? I've documented it heavily should anyone else encounter a disconnect from the results testssl.sh reports with --each-cipher and the expected list. testssl.sh isn't at fault here, and I doubt the issue is common enough to warrant much effort to accommodate by testssl.sh, this issue alone with the insights provided is probably sufficient?

If you would like a public instance. I'll put one up for a limited time, but I'm equally happy just closing the issue ๐Ÿ‘

Original response

I doesn't seem there should be a need for the documentation to specifically warn that testssl.sh doesn't always do things exactly as OpenSSL does. There are many other clients (e.g., Chrome, Edge, Firefox), and they are all different from each other.

All I'm saying is it's a bit misleading both in the docs and terminal output, to claim that --each-cipher is testing N cipher suites against the server, when that's not really what happens.

To report a cipher suite that can be negotiated successfully as not a/v is not helpful. I'm not against that as I can understand why testssl.sh may prefer that for logic, but then the docs should at least convey that not a/v does not actually mean the cipher suite wasn't supported for negotiation, it just wasn't attempted.

In this case, testssl.sh is the client, so the issue isn't with the client.

I understand that. The issue described here would be the same problem if testssl.sh were to offer a cipher suite that it would fail to negotiate with a server that could negotiate the cipher suite successfully on it's end.

As you mentioned with the extra work testssl.sh does to support testing other cipher suites, I'm sure you can understand how a client could run into the same problem if it behaved in such a way. My point here was that the failure scenario would occur regardless of client or server, only one needs to offer a cipher suite it can't actually negotiate.

testssl.sh doesn't "bail early."
If testssl.sh sends the server a list of 50 cipher suites that it supports. If the server supports any one of these 50 cipher suites, then it is supposed to select one of them and continue.
If the negotiation fails, then (in a non-buggy server) that means the server is indicating that it doesn't support any of the 50 cipher suites offered by the client.

Yes, again I understand what you're saying here.

But keep in mind that with client preference it will depend on how you approach negotiating those cipher suites with the server. If you provide a list A-B-D-E-C from testssl.sh and the server is mistakenly configured to offer all of those (but would fail on D), then testssl.sh stops at D.

In your test, absolutely that connection in a real-world scenario won't negotiate pass D from real clients. But if those real clients were to offer their own client preference of A-B-C-D-E, now they're also able to negotiate C before failing. Neither manages to negotiate E.

testssl.sh however (with it's own custom client preference that is not aligned with OpenSSL AFAIK), is now assuming that the remaining cipher suites aren't supported due to the earlier fail on negotiating D.

The error I get with OpenSSL for an unsupported cipher suite by the server (if the client offered only a cipher suite that the server didn't support), and having the client/server negotiate a DHE cipher suite (without the server configured with DH parameters) is the same as I showed output above.

No. There is no concept of the server offer a cipher suite that "is not known during this process."
The server can only choose from the cipher suites offered by the client. Since testssl.sh is the client, it knows all of the cipher suites that it offers.

I was inquiring if testssl.sh / client had any information on what the server supported upfront. If only the server gets a list of what the client supports, that's understandable and is what I've assumed is the case, limiting what testssl.sh can do.

it then sends a message offering the remaining (e.g., 99) cipher suites. It keeps doing this until the server indicates that it doesn't support any of the cipher suites offered.

Right so if testssl.sh offers 50 cipher suites, and the server only supports 10, testssl.sh is only negotiating those 10 correct? The server receives the full list, filters it, then one is chosen by either server or client preference, and testssl.sh omits the successfully negotiated cipher suite while never negotiating the ones the server doesn't offer back.

So, a failed negotiation means that testing is complete, not that testssl.sh is bailing early.

Ideally it would mean it is complete, and we can assume there is no more compatible ciphers.

When there are more cipher suites that are compatible, it's just not with testssl.sh due to it's client preference order and hitting this failure scenario.... that's bailing early. It is not doing what is implied for --each-cipher.

Docs:

-e, --each-cipher checks each of the (currently configured) 370 ciphers via openssl

Terminal output:

Testing 372 ciphers via OpenSSL plus sockets against the server, ordered by encryption strength

# ...

xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH       ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256        not a/v

Given that not a/v is inaccurate to report, when it's entirely dependent upon client preference in this scenario.


Without more information, I wouldn't be able to say why testssl.sh was unable to determine whether the server had its own cipher preference order, but it doesn't mean that testssl.sh knew that the server had a DHE misconfiguration issue.

I mean I have given a way that you can reproduce it fully locally offline. All you need is the compose.yaml + commands I've documented earlier, installing Docker Compose is rather simple/easy, just follow the official docs (Docker CLI or for Windows/macOS go with Docker Desktop).

I could look at setting up a public VPS to run the misconfigured instance with a LetsEncrypt service, but I don't know if there's much point? Any service that can offer a cipher suite to the client/server that would fail during the negotiation step should be sufficient (I've just identified that to be the case with Dovecot and using Docker Compose greatly simplifies that). You could still run testssl.sh locally instead of via container if you preferred that.

I can only assume that however you've implemented the client/server preference detection, that the failure to determine is related to the issue I've reported here. Client detection works perfectly fine once the DHE cipher suites are disabled from the Dovecot server cipher list, or I provide config for DH parameters so that the DHE cipher suites negotiate correctly.

Probably you have logic that first discovers the cipher suites supported, then iterates through that to see if it's in the same order as client preference offered? But instead of mistaking it as server preference there is uncertainty from the results ๐Ÿคทโ€โ™‚๏ธ (perhaps an alternative client preference is attempted but fails to return the same number of cipher suites, for the exact reason I mentioned earlier when client preference differs)


However, I don't think it would make sense to the code in general to try to work around this server bug.

Again. I am not asking for a workaround. I am only raising awareness of this scenario and suggesting testssl.sh offer a more accurate opt-in scan for cipher suite compatibility, rather than the "bail early" strategy that is only reliable for a server that isn't misbehaving due to it's offered cipher list.

More appropriate would be to add a test to the --grease testing (which is not run by default) that would try to detect some specific incorrect behavior when DHE cipher suites are offered.

While DHE was the culprit this time, I think it's more accurate to say it's about any cipher suite that fails to negotiate properly once agreed on. I don't know why that would happen for another cipher suite, but perhaps that's a possibility.

FWIW, there is a software Postfix, that when it doesn't have DH params configured, when paired with OpenSSL 3.0+ it will acquire the DH params (FFDHE RFC 7919 standardized params) from OpenSSL as a fallback. So in that scenario DHE cipher suites still worked, and while I expected Dovecot to not have the fallback, I didn't expect it to still negotiate them in a manner that prevented negotiating additional cipher suites that should have been available (but as noted that concern is a bug with Dovecot).

Thanks a lot @polarathene , also for the offer putting up the instance. I am currently completely absorbed by other matters.

No worries, I think everyone's time is probably better spent elsewhere ๐Ÿ˜…

I'll close the issue. It's probably only worth resolving if other users ever chime in here with a similar experience ๐Ÿ‘