(MFA) Unable to process OTP token

[rlonberg]

I am working with a user in Argentina who is currently unable to verify their OTP token using this tool. The user has tried both Google Authenticator and OneLogin protect, and is not having issues logging into the OneLogin browser. I myself am able to login successfully with this tool.
Here is the user's output:

OneLogin AWS Assume Role Tool
OneLogin Password: 
MFA Required
Authenticate using one of these devices:
-----------------------------------------------------------------------
 0 | Google Authenticator
-----------------------------------------------------------------------
Enter the OTP Token for Google Authenticator: 
176072

The OTP Token was not able to be processed, please introduce a new one: 

We've done some troubleshooting here and I've found that the following line seems to be failing,
https://github.com/onelogin/onelogin-python-aws-assume-role/blob/master/src/aws_assume_role/aws_assume_role.py#L301

To get more information, we've printed out this client object after the client.get_saml_assertion_verifying() call, and the error looks like it might be a json parsing issue on the onelogin-python-sdk side.

{
'client_id': '***************', 
'client_secret': '***************', 
'max_results': 1000, 
'url_builder': <onelogin.api.util.urlbuilder.UrlBuilder object at 0x11345b550>, 
'user_agent': 'onelogin-python-sdk 1.5.0', 
'access_token': '***************', 
'refresh_token': '***************', 
'expiration': datetime.datetime(2020, 8, 25, 20, 51, 15, 552000, tzinfo=tzutc()), 
'error': 500, 
'error_description': 'Expecting value: line 1 column 1 (char 0)', 
'error_attribute': None
}

I've taken a look a the onelogin-python-sdk client object code, specifically the get_saml_assertion_verifying method called by aws-assume-role: https://github.com/onelogin/onelogin-python-sdk/blob/master/src/onelogin/api/client.py#L1632

This error_description Expecting value: line 1 column 1 (char 0) might be the result of a JSON decoding error here https://github.com/onelogin/onelogin-python-sdk/blob/b40b67ca70c74cf0da02da6112c9e531f86bce8f/src/onelogin/api/client.py#L109
.. however, I am guessing this error would likely be resolved upstream of that code execution.

I am opening an issue because this error doesn't seem to be isolated to the user in Argentina. We also have a user in Cape Town experiencing the same error. I myself am having no errors and am based in the US. I'm not sure that these regions are at all relevant to this error, but am adding them for context.

Please let me know if there is other user information I can add to this issue that would help! Thank you.

[rlonberg]

Closing this with a summary of the OneLogin Service Incident Report our team received on August 27, 2020.

What occurred
After moving traffic to a new, faster cluster, one endpoint /api/1/saml_assertion/verify_factor began returning 500 error codes.

Affected systems
[US] ​API
[EU]​ API

Root cause
The /api/1/saml_assertion/verify_factor endpoint wasn’t configured to reset force_logout setting for the authenticating user.

Restorative actions taken
● Deployed fix to restore the impacted endpoint’s normal functionality. Preventive actions taken
● Enhanced testing suite to cover this use case.
● Additional controls implemented around this functionality to prevent future regressions
● Improved monitoring on public authentication API endpoints