[Github action fail] MacOS: "./testssl.sh: line 17665: [[: 0" plus different results for Secure Client-Initiated Renegotiation

Question

[Github action fail] MacOS: "./testssl.sh: line 17665: [[: 0" plus different results for Secure Client-Initiated Renegotiation

Closed this issue 6 months ago · 14 comments

I am currently working on a CI test for MacOS and it looks I encountered a somewhat known problem (see our previous discussion starting here)

Output from yesterday is here: https://github.com/testssl/testssl.sh/actions/runs/14916217628/job/41902331199 . There's often the reference to ./testssl.sh: line 17665: [[: 0.

Running it locally on my Mac doesn't show that, so it seems similar as last time.

As I would love to get CI tests for MacOS working at some point. Can you assist here? PR is #2761 .

Answer 1 · 2025-05-09T09:37:01.000Z

It seems to be a Bash 3.2 built in echo command bug/misbehavior: in case of the expected broken pipe the errors should go to /dev/null. Could you try with /usr/bin/echo or equivalent on MacOS ?
Could be subshell error propagation too.
Things to try:

--- a/testssl.sh
+++ b/testssl.sh
@@ -17656,7 +17656,7 @@ run_renego() {
                # s_client STDIN too early as the close could come at any time and race with the tear down of s_client.
                # See https://github.com/drwetter/testssl.sh/issues/2590
                # In this case the added iteration is harmless as it will just spin in backgroup
-               for ((i=0; i <= ssl_reneg_attempts; i++ )); do sleep $ssl_reneg_wait; echo R 2>/dev/null; k=0; \
+               for ((i=0; i <= ssl_reneg_attempts; i++ )); do sleep $ssl_reneg_wait; /usr/bin/echo R 2>/dev/null; k=0; \
                    # 0 means client is renegotiating & doesn't return an error --> vuln!
                    # 1 means client tried to renegotiating but the server side errored then. You still see RENEGOTIATING in the output
                    # Exemption from above: server closed the connection but return value was zero
@@ -17665,7 +17665,7 @@ run_renego() {
                          && [[ $(tail -1 $ERRFILE | grep -acE '^(RENEGOTIATING|depth|verify|notAfter)') -eq 1 ]] \
                          && [[ $k -lt 120 ]]; \
                        do sleep $ssl_reneg_wait; ((k++)); if (tail -5 $TMPFILE| grep -qa '^closed'); then break; fi; done; \
-               done) | \
+               done) 2>/dev/null | \
                $OPENSSL_NOTIMEOUT s_client $(s_client_options "$proto $legacycmd $STARTTLS $BUGS -connect $NODEIP:$PORT $PROXY $SNI") >$TMPFILE 2>$ERRFILE &
           pid=$!
           ( sleep $((ssl_reneg_attempts*3+3)) && kill $pid && touch $TEMPDIR/was_killed ) >&2 2>/dev/null &

Answer 2 · 2025-05-09T09:44:29.000Z

Thanks for the fast feedback @Tazmaniac . I'll try later... as said though locally on a Mac I wasn't observing this.

If you want to try in the meantime you can checkout the mac_runner branch.

Answer 3 · 2025-05-15T15:55:36.000Z

Ok, thanks. /usr/bin/echo worked. However I am not tempted to use a fixed coded path here. It's ugly.

Then: The Secure Client-Initiated Renegotiation has a problem on Mac. The status from the PoC mac runner, see https://github.com/testssl/testssl.sh/actions/runs/15020245565/job/42207479176?pr=2761 :

On the Mac itself it loops forever:

Looks to me like a bash 3-related issue. It doesn't occur on Linux with LibreSSL or or MacOS with OpenSSL.

It would be great if you could fix that. FYI. The mac runner is here: https://github.com/testssl/testssl.sh/blob/mac_runner/.github/workflows/unit_tests.yml and a simple run with ./testssl.sh -q -R testssl.shmay suffice. Or using bash3 on Linux.

Answer 4 · 2025-05-15T17:36:25.000Z

I am investigating a fix.
Another option would be to disable builtin echo with 'enable -n echo' on macos.

Answer 5 · 2025-05-15T18:06:22.000Z

Ok,

@@ -17665,7 +17665,7 @@ run_renego() {
                          && [[ $(tail -1 $ERRFILE | grep -acE '^(RENEGOTIATING|depth|verify|notAfter)') -eq 1 ]] \
                          && [[ $k -lt 120 ]]; \
                        do sleep $ssl_reneg_wait; ((k++)); if (tail -5 $TMPFILE| grep -qa '^closed'); then break; fi; done; \
-               done) | \
+               done) 2>/dev/null | \
                $OPENSSL_NOTIMEOUT s_client $(s_client_options "$proto $legacycmd $STARTTLS $BUGS -connect $NODEIP:$PORT $PROXY $SNI") >$TMPFILE 2>$ERRFILE &
           pid=$!
           ( sleep $((ssl_reneg_attempts*3+3)) && kill $pid && touch $TEMPDIR/was_killed ) >&2 2>/dev/null &

Seems to be a valid workaround too.
There is still spurious 'failed to flush stdout' message but without affecting the code behavior or the unit test.

Answer 6 · 2025-05-15T18:27:33.000Z

I am investigating a fix.

👍

Another option would be to disable builtin echo with 'enable -n echo' on macos.

To be honest that seems a bit ugly to me too.

The 2>/dev/null seems only to suppress failed to flush stdout in the github action. The failed to flush stdout doesn't happen on MacOS otherwise.

Answer 7 · 2025-05-19T11:54:19.000Z

Hi @Tazmaniac : did you get anywhere?

The renego thing

Linux: "secure_client_renego","testssl.sh/81.169.166.184","443","OK","not vulnerable","CVE-2011-1473","CWE-310"

vs mac: "secure_client_renego","testssl.sh/81.169.166.184","443","OK","not vulnerable, mitigated","CVE-2011-1473","CWE-310"

seems the most pressing issue as the distributed code gives a different result on MacOS.

Answer 8 · 2025-05-19T12:37:52.000Z

I is very difficult to me to debug for macos. I don't have access to macos conputer/vm.
I'm trying some things with github CI. Will give you updates.

Answer 9 · 2025-05-19T12:47:11.000Z

With the version from my macos branch (https://github.com/Tazmaniac/testssl.sh/tree/macos)

Could you run on macos testssl.sh -R --debug 1 https://81.169.166.184 and give me the debug files 81.169.166.184.run_renego.txt and 81.169.166.184.run_renego.errorlog ?

Answer 10 · 2025-05-19T12:51:32.000Z

Thanks for the update. A couple of choices:

use the t the mac_runner branch with the CI check and restrict it on the only necessary code like prove -v t/61_diff_testsslsh.t (last line) . You can modify the check2run variable to narrow it down further. That would be for the issue "different results for Secure Client-Initiated Renegotiation"
for the "/testssl.sh: line 17665: [[: 0" issue (lower prio as above as it happens "only"for the CI runs) you probably need most of the CI check to get that fixed in testssl.sh.

Alternatively

Get bash 3 from https://ftp.gnu.org/gnu/bash/
if needed (doesn't seem so for this issue): get LibreSSL 3.7.x from here: https://www.libressl.org/releases.html

Answer 11 · 2025-05-19T13:57:34.000Z

Running the test with this patch

@@ -17665,7 +17665,7 @@ run_renego() {
                          && [[ $(tail -1 $ERRFILE | grep -acE '^(RENEGOTIATING|depth|verify|notAfter)') -eq 1 ]] \
                          && [[ $k -lt 120 ]]; \
                        do sleep $ssl_reneg_wait; ((k++)); if (tail -5 $TMPFILE| grep -qa '^closed'); then break; fi; done; \
-               done) | \
+               done) 2>/dev/null | \
                $OPENSSL_NOTIMEOUT s_client $(s_client_options "$proto $legacycmd $STARTTLS $BUGS -connect $NODEIP:$PORT $PROXY $SNI") >$TMPFILE 2>$ERRFILE &
           pid=$!
           ( sleep $((ssl_reneg_attempts*3+3)) && kill $pid && touch $TEMPDIR/was_killed ) >&2 2>/dev/null &

Give me expected result:
https://github.com/Tazmaniac/testssl.sh/actions/runs/15114833717/job/42482757375

And the failed to flush stdout message seems to not be in my code.

Answer 12 · 2025-05-19T14:23:40.000Z

Sigh, can reproduce that. Looks like the /usr/bin/echo you suggested was not a good idea.