ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN

[ikkysleepy]

I keep having an issue with a user's password being refused over and over. I am creating a user via the HTTP API and using AMQPLAIN to verify the user and it works fine for most users. Some user have an issue where all of a sudden they get this error message: ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. I reset their password and all is well until a couple of days later. I looked at the logs and tried different passwords but still have the same issue. I think what is happening is (1) I reset the password to be the same or new and the account works (2) a new users get added and (3) this causes the user who has the password to stop working.

Here is the log

`2019-08-26 06:41:44.538 [info] <0.20017.0> Changing password for 'problem_user@domain.com'
2019-08-26 06:41:44.541 [info] <0.20017.0> Setting user tags for user 'problem_user@domain.com' to []
2019-08-26 06:41:44.560 [info] <0.20025.0> Setting permissions for 'problem_user@domain.com' in 'subscribed' to '.*', '.*', '.*'
2019-08-26 06:45:02.303 [info] <0.20052.0> Changing password for 'problem_user@domain.com'
2019-08-26 06:45:02.305 [info] <0.20052.0> Setting user tags for user 'problem_user@domain.com' to []
2019-08-26 06:45:02.325 [info] <0.20059.0> Setting permissions for 'problem_user@domain.com' in 'subscribed' to '.*', '.*', '.*'
2019-08-26 06:49:24.989 [info] <0.20084.0> TLS server: In state hello at tls_handshake.erl:216 generated SERVER ALERT: Fatal - Protocol Version

2019-08-26 11:25:45.168 [info] <0.20884.0> Creating user 'new_user@domain.com'
2019-08-26 11:25:45.171 [info] <0.20884.0> Setting user tags for user 'new_user@domain.com' to []
2019-08-26 11:25:45.189 [info] <0.20891.0> Setting permissions for 'new_user@domain.com' in 'subscribed' to '.*', '.*', '.*'
2019-08-26 17:42:05.622 [info] <0.21987.0> accepting AMQP connection <0.21987.0> (99.198.138.xx:56171 -> 172.31.44.3:5671)
2019-08-26 17:42:05.691 [info] <0.21987.0> Connection <0.21987.0> (99.198.138.xx:56171 -> 172.31.44.3:5671) has a client-provided name: eddy_rabbit
2019-08-26 17:42:05.757 [info] <0.21987.0> connection <0.21987.0> (99.198.138.xx:56171 -> 172.31.44.3:5671 - eddy_rabbit): user 'existing_good_user@domain.com' authenticated and granted access to vhost 'subscribed'
2019-08-26 18:12:08.598 [warning] <0.21987.0> closing AMQP connection <0.21987.0> (99.198.138.xx:56171 -> 172.31.44.3:5671 - eddy_rabbit, vhost: 'subscribed', user: 'existing_good_user@domain.com'):
client unexpectedly closed TCP connection
2019-08-26 18:12:12.334 [info] <0.22089.0> accepting AMQP connection <0.22089.0> (99.198.138.xx:362xx -> 172.31.44.3:5671)
2019-08-26 18:12:12.402 [info] <0.22089.0> Connection <0.22089.0> (99.198.138.xx:362xx -> 172.31.44.3:5671) has a client-provided name: eddy_rabbit
2019-08-26 18:12:12.483 [info] <0.22089.0> connection <0.22089.0> (99.198.138.xx:362xx -> 172.31.44.3:5671 - eddy_rabbit): user 'existing_good_user@domain.com' authenticated and granted access to vhost 'subscribed'
2019-08-26 18:20:38.092 [info] <0.22122.0> TLS server: In state hello at tls_handshake.erl:216 generated SERVER ALERT: Fatal - Protocol Version

2019-08-26 18:25:28.944 [info] <0.22143.0> Changing password for 'problem_user@domain.com'
2019-08-26 18:25:28.946 [info] <0.22143.0> Setting user tags for user 'problem_user@domain.com' to []
2019-08-26 18:25:28.965 [info] <0.22150.0> Setting permissions for 'problem_user@domain.com' in 'subscribed' to '.*', '.*', '.*'
2019-08-26 18:25:55.860 [info] <0.22186.0> accepting AMQP connection <0.22186.0> (127.0.0.1:33214 -> 127.0.1.1:5671)
2019-08-26 18:25:55.863 [error] <0.22186.0> Error on AMQP connection <0.22186.0> (127.0.0.1:33214 -> 127.0.1.1:5671, state: starting):
AMQPLAIN login refused: user 'problem_user@domain.com' - invalid credentials

I have disabled tlsv1.0 and only support 1.2 and 1.1 so I don't know why the log show a fatal error and or if it is related to the problem. I have Rabbitmq 3.7.17 with Erlang 21.1.1 installed.

Any guidance to how to troubleshot or fix the issue would be great. Thanks

[lukebakken]

Hello and thanks for using RabbitMQ.

Could you please post this to the rabbitmq-users mailing list? Please include RabbitMQ, Erlang and operating system version.

Based on the output you provide it does not look like probled_user ever authenticates correctly. Most likely the password used to create the user and what is being used by an application don't match. You can use rabbitmqctl authenticate_user to test (be sure to quote characters correctly in your shell).

[michaelklishin]

TLS protocol version is orthogonal and is NOT logged for the same connection. Addition of a user cannot change any information for existing users. Use rabbitmqctl authenticate_user and a suitable mechanism for inspecting what credentials your applications attempt to use (including Wireshark, which can decrypt traffic if provided with a private key).

[ikkysleepy]

Thanks for the reply. I am connected now but apparently the password is not valid and you won't be disconnected. I found the issue, which was not related to rabbitmq. I was sending a create account api call for the same user on login of that user, which created a new random password. SInce the user already existed it just updated the password for that user.