[HELP needed] [RabbitMQ 3.13.2, Erlang 26.2.4, opensslv3] STOMP, rabbit_auth_backend_http plugins are failing
Closed this issue · 9 comments
Can any one please help, guid eme whats going wrong and how can i fix it.
I am using rabbitmq 3.13.2 and erlang 26.2.4 built with opensslv3.
I am getting below erros for connections and STOMP, rabbit_auth_backend_http plugins are logging failures:
2024-06-19 15:52:59.707901+05:30 [info] <0.695.0> accepting STOMP connection <0.695.0> (127.0.0.1:47302 -> 127.0.0.1:13777)
2024-06-19 15:52:59.708686+05:30 [warning] <0.695.0> AMQP 0-9-1 client call timeout was 70000 ms, is updated to a safe effective value of 130000 ms
2024-06-19 15:52:59.730803+05:30 [warning] <0.704.0> Description: "Failed to assert middlebox server message"
2024-06-19 15:52:59.730803+05:30 [warning] <0.704.0> Reason: [{missing,{change_cipher_spec,1}}]
2024-06-19 15:52:59.730803+05:30 [warning] <0.704.0>
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> TLS client: In state hello_retry_middlebox_assert at ssl_gen_statem.erl:803 generated CLIENT ALERT: Fatal - Unexpected Message
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> - {unexpected_msg,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> {internal,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> {server_hello,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> {3,3},
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> <<189,34,79,32,65,39,124,139,5,165,81,231,179,101,60,216,220,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 215,126,39,13,114,71,249,107,213,112,93,205,18,117,138>>,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> <<144,69,111,100,76,49,215,152,245,7,220,118,145,182,84,110,175,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 51,233,92,173,248,136,61,106,199,176,87,93,191,170,38>>,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> <<19,1>>,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 0,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> #{server_hello_selected_version =>
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> {server_hello_selected_version,{3,4}},
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> key_share =>
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> {key_share_server_hello,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> {key_share_entry,secp256r1,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> <<4,52,84,3,100,51,88,0,55,31,87,65,78,113,59,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 209,49,77,40,185,155,136,184,41,0,224,149,92,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 136,75,17,175,24,70,120,19,74,12,161,247,119,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 112,232,136,212,6,139,134,183,34,10,103,134,
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> 57,115,255,11,81,55,111,193,47,69,132,113>>}},
2024-06-19 15:52:59.731075+05:30 [notice] <0.704.0> pre_shared_key => undefined}}}}
2024-06-19 15:52:59.748803+05:30 [warning] <0.709.0> Description: "Failed to assert middlebox server message"
2024-06-19 15:52:59.748803+05:30 [warning] <0.709.0> Reason: [{missing,{change_cipher_spec,1}}]
2024-06-19 15:52:59.748803+05:30 [warning] <0.709.0>
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> TLS client: In state hello_retry_middlebox_assert at ssl_gen_statem.erl:803 generated CLIENT ALERT: Fatal - Unexpected Message
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> - {unexpected_msg,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> {internal,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> {server_hello,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> {3,3},
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> <<19,62,218,113,80,213,15,162,166,215,3,38,165,189,51,63,251,25,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 132,4,109,106,174,250,203,21,128,19,87,144,5,128>>,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> <<76,134,214,137,45,161,80,61,56,48,233,177,162,41,247,215,4,97,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 50,176,255,52,229,57,202,132,243,42,162,56,146,99>>,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> <<19,1>>,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 0,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> #{server_hello_selected_version =>
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> {server_hello_selected_version,{3,4}},
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> key_share =>
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> {key_share_server_hello,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> {key_share_entry,secp256r1,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> <<4,99,24,144,166,30,91,151,247,108,208,40,128,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 214,165,132,163,115,81,56,192,127,176,133,250,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 160,123,57,81,147,69,170,251,62,118,213,154,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 73,186,18,21,200,222,88,70,101,47,239,154,17,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 98,193,200,4,236,91,233,91,150,146,107,162,
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> 222,64,157>>}},
2024-06-19 15:52:59.749016+05:30 [notice] <0.709.0> pre_shared_key => undefined}}}}
2024-06-19 15:52:59.750155+05:30 [warning] <0.695.0> STOMP login failed for user 'user': authentication failed
2024-06-19 15:52:59.750225+05:30 [error] <0.695.0> STOMP error frame sent:
2024-06-19 15:52:59.750225+05:30 [error] <0.695.0> Message: "Bad CONNECT"
2024-06-19 15:52:59.750225+05:30 [error] <0.695.0> Detail: "Access refused for user 'user'"
2024-06-19 15:52:59.750225+05:30 [error] <0.695.0> Server private detail: none
...
024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> TLS client: In state hello_retry_middlebox_assert at ssl_gen_statem.erl:803 generated CLIENT ALERT: Fatal - Unexpected Message
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> - {unexpected_msg,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> {internal,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> {server_hello,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> {3,3},
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> <<18,46,179,140,86,49,79,60,199,127,85,75,217,17,198,115,210,60,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 203,240,173,116,36,132,170,40,214,56,147,130,58,235>>,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> <<237,107,64,165,251,152,203,57,233,250,6,239,185,115,32,22,131,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 30,46,79,23,248,38,123,155,120,154,19,197,3,246,162>>,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> <<19,1>>,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 0,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> #{server_hello_selected_version =>
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> {server_hello_selected_version,{3,4}},
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> key_share =>
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> {key_share_server_hello,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> {key_share_entry,secp256r1,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> <<4,108,49,1,8,238,21,193,244,212,252,195,195,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 127,66,186,130,190,14,226,52,171,238,83,84,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 211,214,131,247,84,33,215,186,147,143,161,89,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 160,147,74,164,234,219,34,117,24,225,224,239,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 10,230,125,176,110,147,243,178,194,180,203,
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> 121,84,54,98,162>>}},
2024-06-19 15:53:02.221607+05:30 [notice] <0.720.0> pre_shared_key => undefined}}}}
2024-06-19 15:53:02.248266+05:30 [warning] <0.725.0> Description: "Failed to assert middlebox server message"
2024-06-19 15:53:02.248266+05:30 [warning] <0.725.0> Reason: [{missing,{change_cipher_spec,1}}]
2024-06-19 15:53:02.248266+05:30 [warning] <0.725.0>
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> TLS client: In state hello_retry_middlebox_assert at ssl_gen_statem.erl:803 generated CLIENT ALERT: Fatal - Unexpected Message
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> - {unexpected_msg,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> {internal,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> {server_hello,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> {3,3},
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> <<253,1,67,246,248,124,110,44,60,149,189,219,103,19,20,7,105,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 165,90,203,74,220,22,13,6,249,251,11,161,162,55,134>>,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> <<201,36,234,29,154,249,39,172,216,54,65,13,57,219,155,37,71,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 255,137,142,55,100,65,15,108,110,163,113,28,228,233,32>>,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> <<19,1>>,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 0,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> #{server_hello_selected_version =>
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> {server_hello_selected_version,{3,4}},
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> key_share =>
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> {key_share_server_hello,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> {key_share_entry,secp256r1,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> <<4,78,203,135,153,247,150,225,13,48,32,190,128,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 253,252,140,222,232,111,209,193,115,94,40,197,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 131,76,159,41,202,18,253,75,219,36,158,245,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 144,37,8,111,211,26,17,27,177,246,151,11,79,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 242,119,12,46,172,194,174,187,105,60,112,92,
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> 87,84,217>>}},
2024-06-19 15:53:02.248876+05:30 [notice] <0.725.0> pre_shared_key => undefined}}}}
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> Error on AMQP connection <0.715.0> (:49959 -> :13781, state: starting):
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> PLAIN login refused: rabbit_auth_backend_http failed authenticating 1718869588636_5dc46b3b-2e4d-44e4-a05f-2516466b5aff: {failed_connect,
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> [{to_address,
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> {"",
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> 1556}},
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> {inet,
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> [inet],
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> {tls_alert,
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> {unexpected_message,
2024-06-19 15:53:02.252532+05:30 [error] <0.715.0> "TLS client: In state hello_retry_middlebox_assert at ssl_gen_statem.erl:803 generated CLIENT ALERT: Fatal - Unexpected Message\n {unexpected_msg,\n {internal,\n {server_hello,\n {3,3},\n <<253,1,67,246,248,124,110,44,60,149,189,219,...>>,\n <<201,36,234,29,154,249,39,172,216,54,65,...>>,\n <<19,1>>,\n 0,\n #{server_hello_selected_version =>\n {server_hello_selected_version,{3,4}},\n key_share =>\n {key_share_server_hello,\n {key_share_entry,secp256r1,<<4,78,...>>}},\n pre_shared_key => undefined}}}}"}}}]}
2024-06-19 15:53:02.253549+05:30 [info] <0.715.0> closing AMQP connection <0.715.0> (:49959 -> :13781)
Well the problem is that the server you are connecting to is not adhering to the spec on how middlebox mode should behave.
Probably the easiest fix is to disable middle-box mode as you probably do not need it anyway. You can do this by
giving the option {middlebox_comp_mode, false} . The intent with it being on by default was to allow more out of the box compatibility but in reality I think that there are more flaky TLS-1.3 implementations then middleboxes causing problems. We are considering some kind of relaxed middle-box mode but we do feel that we want to adhere to the specification by default.
Well the problem is that the server you are connecting to is not adhering to the spec on how middlebox mode should behave. Probably the easiest fix is to disable middle-box mode as you probably do not need it anyway. You can do this by giving the option
{middlebox_comp_mode, false}. The intent with it being on by default was to allow more out of the box compatibility but in reality I think that there are more flaky TLS-1.3 implementations then middleboxes causing problems. We are considering some kind of relaxed middle-box mode but we do feel that we want to adhere to the specification by default.
Thanks,
I am checking that, And am I suppose to add option under any specific section?
Can you please share the exact location, I have added it in
{rabbitmq_auth_backend_http,
[{http_method, post},
{user_path, "some value"},
{vhost_path, "some value"},
{resource_path, "some value"},
{topic_path, "some value"},
{ssl_options, [{cacertfile, "some value"},
{verify, verify_peer},
%%{server_name_indication, "some value"},
{depth, 5},
{middlebox_comp_mode, false}]},
{middlebox_comp_mode, false}
]},
But its still giving me errors for both the plugins
It is a ssl_option to be set on the client.
It is a ssl_option to be set on the client.
okay, i tried to give it in each existing sections' ssl_option, but didnt work, trying again. Thanks, I will kepe you updated with result
You could try verifying your options in an erlang shell calling ssl:connect/3 first and then supply them via appropriate Rabbit configuration.
Just to make a note, this issue arising only if I am disbaling FIPS. If I am enabling FIPS. This issue is not getting reproduced.
You could try verifying your options in an erlang shell calling ssl:connect/3 first and then supply them via appropriate Rabbit configuration.
May be its working, I will try more do more testing and will update this ticket
The issue is fixed.
Thanks IngelaAndin for providing the solution.