Login fails
viaConBodhi opened this issue · 35 comments
upgraded to latest package today to see if that would fix and still having issues...all was running fine last Friday and today I'm getting a
400 Client Error: Bad Request for url: https://api.robinhood.com/oauth2/token/
log into Fast_Arrow
2 client = Client(username=XXXXX, password=XXXXXX)
----> 3 client.authenticate()
4 #login_oauth2
5 # client = Client.login_oauth2(self,username=XXXXX, password=XXXXX)
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in authenticate(self)
32 '''
33 if "username" in self.options and "password" in self.options:
---> 34 self.login_oauth2(self.options["username"], self.options["password"], self.options.get('mfa_code'))
35 elif "access_token" in self.options and "refresh_token" in self.options:
36 self.access_token = self.options["access_token"]
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in login_oauth2(self, username, password, mfa_code)
117 data['mfa_code'] = mfa_code
118 url = "https://api.robinhood.com/oauth2/token/"
--> 119 res = self.post(url, payload=data, retry=False)
120
121 if res is None:
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in post(self, url, payload, retry)
78 attempts += 1
79 if res.status_code in [400]:
---> 80 raise e
81 elif retry and res.status_code in [403]:
82 self.relogin_oauth2()
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in post(self, url, payload, retry)
70 try:
71 res = requests.post(url, headers=headers, data=payload, timeout=15, verify=self.certs)
---> 72 res.raise_for_status()
73 if res.headers['Content-Length'] == '0':
74 return None
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\requests\models.py in raise_for_status(self)
938
939 if http_error_msg:
--> 940 raise HTTPError(http_error_msg, response=self)
941
942 def close(self):
@wccramer I see you closed this issue. Did it work for you?
@westonplatter No...I messed up my prior post and started a new. Found the following thread somewhere else on this which means your code most likely is affected for others (like me). Looks like the recent RH update has affected others too but not all. Nice work on the code base though...been very helpful up till when login stopped working. Let me know if you find a solution as I'm keeping an eye out for those with more skills to help provide some options...no pun intended.
@wccramer thanks for the context. Won't have time to look at it for a while. Happy to look at a PR.
@westonplatter If/when I get can get something together I'll send your way.
Also having this error as of a few hours ago, everything was working well last night. (~ 12 hours ago)
Same as @halessi.
Worked perfectly yesterday and last night and then as of this morning I can not access anything.
Checkout the link I posted above for the thread from Jamonek. There are some folks who have a solution but the code is mostly in C#/Java but there is some python. No fully updated code base but looks like you should be able to build out a solution with what has been provided via the discussions. RH has added SMS/email verification so login has changed. Looks like they are rolling it out in phases though.
@Chenzoh12 Haven't tested but it looks like you can opt to have an email sent then, if using Google API, use some reg X and email filtering to capture the # and add to the variable. A few steps but seems like it should work.
Utilizing what they did over at the Robinhood repo linked earlier, I replaced the first few lines of login_oauth2 with:
`if self.device_token == None:
self.device_token = self.GenerateDeviceToken()
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
"password": password,
"username": username,
"challenge_type": 'sms',
"device_token": self.device_token
}`
with GenerateDeviceToken (also written in the other thread) being:
`def GenerateDeviceToken():
rands = []
for i in range(0,16):
r = random.random()
rand = 4294967296.0 * r
rands.append((int(rand) >> ((3 & i) << 3)) & 255)
hexa = []
for i in range(0,256):
hexa.append(str(hex(i+256)).lstrip("0x").rstrip("L")[1:])
id = ""
for i in range(0,16):
id += hexa[rands[i]]
if (i == 3) or (i == 5) or (i == 7) or (i == 9):
id += "-"
return id`
And I get an SMS text. Going to swap to email and see if I can somehow do what @wccramer suggested.
@halessi thanks for that! Let me know if it works out...caught up on day job stuff so I can't play with it now.
Alright, I've managed to get it working off of what was said in the other thread. The initial post for authenticating returns a "challenge", of which it's necessary to save the challenge id. If you have that, you can use the verification code retrieved via the below function (note, user & pass are gmail login info) if you submitted the original post with {'challenge_type': 'email'}:
`def fetch_mfa_code(self):
username, password = user, pass
obj = imaplib.IMAP4_SSL('imap.gmail.com')
obj.login(username, password)
obj.select()
cutoff = (datetime.today() - timedelta(minutes = 2)).strftime('%d-%b-%Y')
typ, data = obj.search(None,
'(SINCE %s) (FROM "notifications@robinhood.com")' % (cutoff,))
email_ids = data[0].decode().split(' ')
email = obj.fetch(email_ids[-1], '(UID BODY[TEXT])')
parsed = email[1][0][1].decode().split('r')[9]
mfa_code = [int(s) for s in re.findall(pattern = r'\d+', string = parsed)]
assert len(str(mfa_code[0])) == 6, print('ERROR: failed to parse mfa code.')
return mfa_code[0]`
Use this verification code alongside the saved challenge id to submit another post:
`if mfa_code is not None and id is not None:
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(id)
payload = {'response' : mfa_code}
res = self.post(url, payload = payload, retry = False)`
Res['status'] should be validated, and you're good! I can make a PR if @westonplatter wants, but I'm certain my solution isn't exactly the best approach.
Proper! good work @halessi ! Looks like you got the regX figured. Can't wait to try this. Not sure if there are too many other approaches around this SMS/email thing so looks like you got a working model without having to manually enter numbers from a SMS.
Ran into issues with gmail running the code above but looks like should work. When/If you all get a google API email setup (not hard and plenty of online support) the code below should help you parse the text from RH. The code still needs to filter based on datetime (creating an input by capturing the datetime prior to RH sending the email then filtering gmail based on that time) but the code basically outputs a data frame on outputDF so you can filter based on time. Make sure when you create the credentials.json file (the file google provides you with all the password things) you save it using this name as it is referenced in the code. The google API is not hard to setup but may take some time depending on your level. Once setup it is pretty straight forward and code is reusable (which is what I've used below).
`'''
For the API varificationSP...Function still needs to take in a datetime to determine a file that is later then the
datetime input so that a record can be accessed that is the most recent
and return the number which is being provided below
Uses the Google Email API to take in emails from Robinhood, checks a listing of prior processed emails,
and returns a data frame with emails that have not been processed.
'''
from future import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
import dateutil.parser as parser
import base64
import email
from apiclient import errors
import pandas as pd
import re
If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']
def spitDigits():
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('gmail', 'v1', credentials=creds)
query = "from:notifications@robinhood.com"
# Call the Gmail API
results = service.users().messages().list(userId='me', q=query, labelIds = ['INBOX']).execute()
labels = results.get('labels', [])
messages = results.get('messages', [])
outputDF = pd.DataFrame()
if not messages:
print("No messages found.")
else:
for message in messages:
msg = service.users().messages().get(userId='me', id=message['id']).execute()
payld = msg['payload'] # get payload of the message
headr = payld['headers'] # get header of the payload
subject_dict = { }
received_dict = { }
time_dict = { }
sender_dict = { }
messageId_dict = { }
for one in headr: # getting the Subject
if one['name'] == 'Subject':
msg_subject = one['value']
subject_dict['Subject'] = msg_subject
#print(subject_dict)
else:
pass
for two in headr: # getting the date
#print(two)
if two['name'] == 'Date':
msg_date = two['value']
date_parse = (parser.parse(msg_date))
received_dict['Received'] = str(date_parse)
else:
pass
for three in headr: # getting the Sender
if three['name'] == 'From':
msg_from = three['value']
sendList = []
sendList.append(msg_from)
sender_dict['Sender'] = sendList
#print(sender_dict)
else:
pass
#this returns the entire message
for pay in msg['payload']['parts']:
msg_str = base64.urlsafe_b64decode(pay['body']['data'].encode('UTF-8'))
messageId_dict['Message ID'] = msg['id']
d= {'messageID':msg['id'],'subject':msg_subject,'email_Received':date_parse \
,'sender':sendList,'snippet':msg['snippet'],'message':msg_str}
df = pd.DataFrame(data=d)
df['subject'] = df['subject'].astype(str).str.lower()
df['sender'] = df['sender'].astype(str).str.lower()
df['snippet'] = df['snippet'].astype(str).str.lower()
df['message'] = df['message'].astype(str).str.lower()
outputDF = outputDF.append(df)
outputDF = outputDF[outputDF['subject']=='your email verification code']
#print(outputDF)
pattern = "\<h3\>\d\d\d\d\d\d\<\/h3\>"
sub = ""
for i in re.findall(pattern, outputDF['message'].iloc[0]):
sub = re.sub(r'\<h3\>|\<\/h3\>','',i)
return(sub)`
Just noticed for some reason the first part of my code is pasting funny...I'll try to get something cleaned up on my profile for easy access for those that may have trouble pulling/editing.
I am fairly new to coding especially with Python/ Python3, but I have found that you will always want the last email from RH and using gmail API quickstart I am able to get the challenge response from my gmail using the below:
def gmail_mfa_code():
creds = None
user_id = 'email@gmail.com'
query = 'from:notifications@robinhood.com'
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('gmail', 'v1', credentials=creds)
response = service.users().messages().list(userId=user_id, q=query).execute()
messages = []
if 'messages' in response:
messages.extend(response['messages'])
while 'nextPageToken' in response:
page_token = response['nextPageToken']
response = service.users().messages().get(userId=user_id, q=query, pageToken=page_token).execute()
messages.extend(response['messages'])
message_id = messages[0]['id']
message = service.users().messages().get(userId=user_id, id=message_id).execute()
challenge_response = message['snippet'][94:100]
print(challenge_response)
return challenge_response
However, I am struggling with posting that challenge code to clear the actual challenge. Any advice?
@Chenzoh12 @halessi
Chenzoh12...finally got some time to catch up on this but seems I'm still getting the 400 message but I'm also getting emailed the pin only with a delay which is weird. I can't grab the "challenge" without the right return so I can't finish the build. Also...for some reason it worked once and then stopped which is even more weird. I think it may be related to the way I'm posting the "data" that has been suggested. One of comments from the other feed notes HTTP/2 is needed to make the post. Are you all using another library to make the posts as it looks like the requests library does not support HTTP/2.
@wccramer @Chenzoh12 Below is my code for the post, where I currently have the logic for dealing with authentication. I've also shared the code I have for GenerateDeviceToken and fetching the mfa_code, SEE MY POST ABOVE. As it stands, this works every time. I added the while loop because Robinhood is remarkably inconsistent in how long it takes to send the 2FA email out (sometimes < 1 second, others > 10).
NOTE: if you want to fetch the mfa_code using my function above, you must configure your Google account settings to allow for less secure connections. Otherwise you can fiddle around with the Google API, I just couldn't be bothered. Associated Robinhood 2FA with a non-important Gmail so if it's ever compromised as a result of changing security settings, no biggie.
ALSO: it is important you configure the "data" dictionary in login_oauth2 to be the following:
`if self.device_token == None:
self.device_token = self.GenerateDeviceToken()
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
"password": password,
"username": username,
"challenge_type": 'email',
"device_token": self.device_token
}`
As for how submitting the challenge works, you have to save the challenge_id from the original post and insert it:
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(res.json()['challenge']['id'])
and also add it to the headers:
headers['X-ROBINHOOD-CHALLENGE-RESPONSE-ID'] = challenge_res.json()['id'].
Obviously having the 2FA logic inside post is less than ideal -- I'll move it and make a PR soon.
`def post(self, url=None, payload=None, retry=True):
'''
Execute HTTP POST
'''
headers = self._gen_headers(self.access_token, url)
attempts = 1
while attempts <= HTTP_ATTEMPTS_MAX:
try:
res = requests.post(url, headers=headers, data=payload,
timeout=15, verify=self.certs)
res.raise_for_status()
if res.headers['Content-Length'] == '0':
return None
else:
return res.json()
except requests.exceptions.RequestException as e:
attempts += 1
if res.status_code in [400, 429]:
if res.json()['challenge']:
''' HANDLE CHALLENGE 2FA '''
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(res.json()['challenge']['id'])
validated = False
while validated == False:
time.sleep(3.5) # wait until the email has been sent
challenge_res = requests.post(url, headers = headers,
data = {'response' : self.fetch_mfa_code()}, timeout = 15, verify = self.certs)
try:
headers['X-ROBINHOOD-CHALLENGE-RESPONSE-ID'] = challenge_res.json()['id']
validated = True
except KeyError:
validated = False
''' TRY TO FETCH REFRESH/ACCESS TOKENS '''
url = "https://api.robinhood.com/oauth2/token/"
challenge_res = requests.post(url, headers = headers,
data = payload, timeout = 15, verify = self.certs)
#print('Response from submitting 2nd challenge code: {}'.format(json.dumps(challenge_res.json(), indent = 4)))
assert challenge_res is not None, print('Multi-factor challenge failed.')
return challenge_res.json()
else:
raise e
elif retry and res.status_code in [403]:
self.relogin_oauth2()`
@halessi Thanks for sharing your solution. Could you post a copy of your entire client.py file? Would make it a lot easier to debug.
@pratik-r Find below. Haven't had time to fix it yet, but it gets stuck in the while loop ~25% of the time (whoops).
Another cool (and necessary!) enhancement would involve storing the device_token that is validated for use in future logins. I believe they said it remains valid for ~24 hours over in the other Robinhood thread.
import imaplib
import os
import random
import time
import re
import json
from datetime import datetime, timedelta
import requests
from fast_arrow.exceptions import AuthenticationError, NotImplementedError
from fast_arrow.resources.account import Account
from fast_arrow.resources.user import User
from fast_arrow.util import get_last_path
CLIENT_ID = "c82SH0WZOsabOXGP2sxqcj34FxkvfnWRZBKlBjFS"
HTTP_ATTEMPTS_MAX = 2
class Client(object):
def __init__(self, **kwargs):
self.options = kwargs
self.account_id = None
self.account_url = None
self.access_token = None
self.refresh_token = None
self.mfa_code = None
self.device_token = None
self.scope = None
self.authenticated = False
self.certs = os.path.join(os.path.dirname(__file__), 'ssl_certs/certs.pem')
def authenticate(self):
'''
Authenticate using data in `options`
'''
if "username" in self.options and "password" in self.options:
self.login_oauth2(self.options["username"], self.options["password"], self.options.get('mfa_code'))
elif "access_token" in self.options and "refresh_token" in self.options:
self.access_token = self.options["access_token"]
self.refresh_token = self.options["refresh_token"]
self.__set_account_info()
else:
self.authenticated = False
return self.authenticated
def get(self, url=None, params=None, retry=True):
'''
Execute HTTP GET
'''
headers = self._gen_headers(self.access_token, url)
attempts = 1
while attempts <= HTTP_ATTEMPTS_MAX:
try:
res = requests.get(url, headers=headers, params=params, timeout=15, verify=self.certs)
res.raise_for_status()
return res.json()
except requests.exceptions.RequestException as e:
attempts += 1
if res.status_code in [400]:
raise e
elif retry and res.status_code in [403]:
self.relogin_oauth2()
def post(self, url=None, payload=None, retry=True):
'''
Execute HTTP POST
'''
headers = self._gen_headers(self.access_token, url)
attempts = 1
while attempts <= HTTP_ATTEMPTS_MAX:
try:
res = requests.post(url, headers=headers, data=payload,
timeout=15, verify=self.certs)
res.raise_for_status()
if res.headers['Content-Length'] == '0':
return None
else:
return res.json()
except requests.exceptions.RequestException as e:
attempts += 1
if res.status_code in [400, 429]:
if res.json()['challenge']:
''' HANDLE CHALLENGE 2FA '''
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(res.json()['challenge']['id'])
validated, trys = False, 0
while validated == False:
trys += 1; print('Trys: {}.'.format(trys))
time.sleep(5) # wait until the email has been sent
challenge_res = requests.post(url, headers = headers,
data = {'response' : self.fetch_mfa_code()}, timeout = 15, verify = self.certs)
try:
headers['X-ROBINHOOD-CHALLENGE-RESPONSE-ID'] = challenge_res.json()['id']
validated = True
except KeyError:
print(json.dumps(challenge_res.json(), indent = 4))
validated = False
''' TRY TO FETCH REFRESH/ACCESS TOKENS '''
url = "https://api.robinhood.com/oauth2/token/"
challenge_res = requests.post(url, headers = headers,
data = payload, timeout = 15, verify = self.certs)
#print('Response from submitting 2nd challenge code: {}'.format(json.dumps(challenge_res.json(), indent = 4)))
assert challenge_res is not None, print('Multi-factor challenge failed.')
return challenge_res.json()
else:
raise e
elif retry and res.status_code in [403]:
self.relogin_oauth2()
def _gen_headers(self, bearer, url):
'''
Generate headders, adding in Oauth2 bearer token if present
'''
headers = {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
}
if bearer:
headers["Authorization"] = "Bearer {0}".format(bearer)
if url == "https://api.robinhood.com/options/orders/":
headers["Content-Type"] = "application/json; charset=utf-8"
return headers
def GenerateDeviceToken():
rands = []
for i in range(0,16):
r = random.random()
rand = 4294967296.0 * r
rands.append((int(rand) >> ((3 & i) << 3)) & 255)
hexa = []
for i in range(0,256):
hexa.append(str(hex(i+256)).lstrip("0x").rstrip("L")[1:])
id = ""
for i in range(0,16):
id += hexa[rands[i]]
if (i == 3) or (i == 5) or (i == 7) or (i == 9):
id += "-"
return id
def fetch_mfa_code(self):
username, password = user, pass
obj = imaplib.IMAP4_SSL('imap.gmail.com')
obj.login(username, password)
obj.select()
cutoff = (datetime.today() - timedelta(minutes = 2)).strftime('%d-%b-%Y')
typ, data = obj.search(None,
'(SINCE %s) (FROM "notifications@robinhood.com")' % (cutoff,))
email_ids = data[0].decode().split(' ')
email = obj.fetch(email_ids[-1], '(UID BODY[TEXT])')
parsed = email[1][0][1].decode().split('r')[9]
mfa_code = [int(s) for s in re.findall(pattern = r'\d+', string = parsed)]
#assert len(str(mfa_code[0])) == 6
if len(str(mfa_code[0])) != 6: mfa_code = '0' + str(mfa_code) # HACK: fix
return mfa_code[0]
def login_oauth2(self, username, password, mfa_code=None, id=None):
'''
Login using username and password
'''
self.username = username
self.password = password
if self.device_token == None:
self.device_token = self.GenerateDeviceToken()
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
"password": password,
"username": username,
"challenge_type": 'email',
"device_token": self.device_token
}
url = "https://api.robinhood.com/oauth2/token/"
res = self.post(url, payload = data, retry = False)
if res is None:
if mfa_code is None:
msg = "Client.login_oauth2(). Could not authenticate. Check username and password."
raise AuthenticationError(msg)
else:
msg = "Client.login_oauth2(). Could not authenticate. Check username and password, and enter a valid MFA code."
raise AuthenticationError(msg)
elif res.get('mfa_required') is True:
msg = "Client.login_oauth2(). Could not authenticate. MFA is required."
raise AuthenticationError(msg)
self.access_token = res["access_token"]
self.refresh_token = res["refresh_token"]
self.mfa_code = res["mfa_code"]
self.scope = res["scope"]
self.__set_account_info()
return self.authenticated
def __set_account_info(self):
account_urls = Account.all_urls(self)
if len(account_urls) > 1:
msg = "fast_arrow 'currently' does not handle multiple account authentication."
raise NotImplementedError(msg)
elif len(account_urls) == 0:
msg = "fast_arrow expected at least 1 account."
raise AuthenticationError(msg)
else:
self.account_url = account_urls[0]
self.account_id = get_last_path(self.account_url)
self.authenticated = True
def relogin_oauth2(self):
'''
(Re)login using the Oauth2 refresh token
'''
url = "https://api.robinhood.com/oauth2/token/"
data = {
"grant_type": "refresh_token",
"refresh_token": self.refresh_token,
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
}
res = self.post(url, payload=data, retry=False)
self.access_token = res["access_token"]
self.refresh_token = res["refresh_token"]
self.mfa_code = res["mfa_code"]
self.scope = res["scope"]
def logout_oauth2(self):
'''
Logout for given Oauth2 bearer token
'''
url = "https://api.robinhood.com/oauth2/revoke_token/"
data = {
"client_id": CLIENT_ID,
"token": self.refresh_token,
}
res = self.post(url, payload=data)
if res == None:
self.account_id = None
self.account_url = None
self.access_token = None
self.refresh_token = None
self.mfa_code = None
self.scope = None
self.authenticated = False
return True
else:
raise AuthenticationError("fast_arrow could not log out.")
@halessi thanks for passing this along...haven't run it yet but looks like you were pulling the 'challenge' from the 400 return which I didn't think was possible with that return . I'll let you know if I have challenges with this. I'm getting emails with access code around the same deltas you described so looks like, at least, that's consistent. Since I was getting 400 codes I thought something was wrong. If I can get this working I'm thinking I could use a stamp = datetime.now when the function initiates and this could be passed to the function for the email processing via a loop (as needed to deal with the email sending delta) so the filter can be based on the time the original function was called. Capturing the return challenge could also be passed to the email function to output a final send for complete login.
@halessi I just tried out your code, and I can confirm this works. I just had to change a few lines in the get_mfa_code function. Apparently my parser doesn't work the same way as yours. But it definitely works. Thanks again.
I got it working by adding a device ID from a computer that was already logged in....
Used the latest code from the master branch, with the below modification with no issues....
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"device_token": DEVICE_ID,
"expires_in": 86400,
"password": password,
"username": username
@jwschmo I just tried it using my desktop's device ID. I logged in manually through the website first, but I'm still getting the same 400 error. I got the device ID from my system settings. What device ID did you use?
This is showing closed to me. Is that correct?
Edit-- Nevermind. I see #86 open. Just got confused with all the updates going here.
@halessi you rock...I was able to integrate the google API and now have access. I was stuck at the post even though I was getting an email with the code and now I'm catching the challenge. I owe you a beer. Good looking out.
First off, hats off to all the people who contributed to this conversation. Thanks for your comments, experimentation, and feedback. I sat down this morning to get things working for myself and leveraged so much of the above conversation. Thank you: @viaConBodhi @halessi @Chenzoh12 @pratik-r @jwschmo @stewood.
Here's my attempt to summarize the conversation and add my own findings (please, correct me if I'm missing something and I'll correct/edit this list),
- the api allows users to authenticate with and without MFA (mutli factor authentication)
- the api expects that a
device_tokenis present in the HTTP POST body - if the included
device_tokenhas not been previously authenticated, Robinhood will require confirmation via Email or Sms. - currently, fast_arrow does not have a solution for handing the
device_tokenconfirmation process.
My goal is to come up with a simple approach to handling the device_token confirmation process so that authentication "works out of the box" without issues for all users. Here are a couple approaches that came to mind. There are likely many more.
1. Adjust the fast_arrow/client.Client to interactively allow users to input the device_token confirmation code. Pros: minimal code changes to fast_arrow; relatively easy to code up. Cons: fast_arrow changes from being a client and becomes an interactive, shell client library.
2. Move authentication out of fast_arrow into a supporting library. Pros: reduces complexity and helps fast_arrow focus on doing relatively few things exceptionally well; really opens the door to providing detailed implementations for automated device_token fetching. Cons: adds another codebase to develop and upkeep.
My preference is option 2; I've got some free cycles the next couple days and will share a simple example library. Once the community has gotten a chance to give it a try, I'd love feedback. However, this proposed solution my not meet everyone's needs so totally open considering other options.
Reopening in order to track / keep notes on solving the device_token issue that keeps authentication from working right out of the box.
My vote is also for option 2. Authentication is a separate and now "more involved" function and depending on the use cases and interfaces people are using option 1 could be a painted corner. Option 2, at least from what I've seen, will still require more user configuration (email or SMS capture) which will relinquish the whole "turn-key" approach but them's the breaks to meet the security requirement. The email approach has been working for me but I still have a few small bugs that I'm willing to live with as I work on other more interesting items. Authentication is the first step in my workflow so if it fails I simply rerun and it works. Not production quality but worthy for what I've knocking out. Thanks again for continuing to support this library as I've found it very resourceful.
Regarding approach 2:
Could we attempt to re-use the same auth.data file already generated by anilshanbhag's RobinhoodShell (following auto-fetching its email verification code as needed)?
So as to try to maintain the same login session across separate but similar RH projects:
{
"auth_token": "eyJ0eSUzI1NiJ9.eyJleHAiOjE1NjM4NDY5MTgsInRva2VuIjoiZzRhVFdzV0owc09HTVB0TUsXAiOiJKV1QiLCJhbGciOiJzdFg4b0pVT1lFNEp1IiwidXNlcl9pZCI6ImI4ZjgxNDg0LTYxODgtNDFiZi1iZGI4LTJlNzhjNTUxMWRlNCIsIm9wdGlvbnMiOnRydWUsImxldmVsMl9hY2Nlc3MiOnRydWV9.Wzt85kF2OWM0QrK7U74LCXMMsMrJJeAFNZ6Xj8sgDpg8i2zG4YYGA7nR3v3C_uQPdAqO8ttPBsRsqayBUqNet6LAKbgVwmvbOarsFM-iuYkLCZeuCftGxIEH7toJYsir835r6djXdso8GfZjLLCbu_vfou6dhH4Dmwq4Gwev3nefOxw7ns1dxZLvuyNM7neaXvu6eQrZckI-1TM_OzvxkfQUTYK-B3x1whIh0qplIcvLQVA9o2JUkCgV4UTmfr88-r0GyzRCGhIHqu0n-LC_RKKoQZifPJ6Fd7r0s4eXn3lIDAYxur4VPinxomMJJpexFb9C_HML0WSIlcchNduhQw",
"device_token": "8e48a02a-9dc4-7d0d-dbea-be40a1c9c206",
"refresh_token": "NoM3RJ0DIThKE3ZdzMDa0a9RbX988X"
}
@klepsydra funny you should mention that as I've been thinking about building out a small tool for this exact topic but have been caught up on other stuff and limping along with something I've hot-wired together. I've been authenticating first with the Fast_Arrow and then passing this auth into a modified "login" for the Jamonek Robinhood package as there are some functions in this library that are not in Fast_Arrow. This same approach may work for your use case or could be incorporated into more general Auth Method for Opt 2
Sorry I know this maybe out of scope of the Fast_Arrow but maybe still in scope with a general Auth Method for opt 2.
Passing along these keys is working for me like the following.
Auth with Fast_Arrow using
client = Client(username="X", password="X")
client.authenticate()
Pass the keys to Jamonek
my_trader = Robinhood()
logged_in = my_trader.login(client)
logged_in
The Jamonek modify is below. I've included the entire logging method just encase.
` ###########################################################################
# Logging in and initializing
###########################################################################
def __init__(self):
self.session = requests.session()
self.session.proxies = getproxies()
self.headers = {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5",
"Content-Type": "application/x-www-form-urlencoded; charset=utf-8",
"X-Robinhood-API-Version": "1.0.0",
"Connection": "keep-alive",
"User-Agent": "Robinhood/823 (iPhone; iOS 7.1.2; Scale/2.00)"
}
self.session.headers = self.headers
self.device_token = ""
self.challenge_id = ""
def login_required(function): # pylint: disable=E0213
""" Decorator function that prompts user for login if they are not logged in already. Can be applied to any function using the @ notation. """
def wrapper(self, *args, **kwargs):
if 'Authorization' not in self.headers:
self.auth_method()
return function(self, *args, **kwargs) # pylint: disable=E1102
return wrapper
def GenerateDeviceToken(self):
rands = []
for i in range(0,16):
r = random.random()
rand = 4294967296.0 * r
rands.append((int(rand) >> ((3 & i) << 3)) & 255)
hexa = []
for i in range(0,256):
hexa.append(str(hex(i+256)).lstrip("0x").rstrip("L")[1:])
id = ""
for i in range(0,16):
id += hexa[rands[i]]
if (i == 3) or (i == 5) or (i == 7) or (i == 9):
id += "-"
self.device_token = id
def get_mfa_token(self, secret):
intervals_no = int(time.time())//30
key = base64.b32decode(secret, True)
msg = struct.pack(">Q", intervals_no)
h = hmac.new(key, msg, hashlib.sha1).digest()
o = h[19] & 15
h = '{0:06d}'.format((struct.unpack(">I", h[o:o+4])[0] & 0x7fffffff) % 1000000)
return h
def login(self,*args):
for i in args:
testing = i.access_token
username = i.username
password = i.password
device_token = i.device_token
auth_token = i.access_token
refresh_token = i.refresh_token
self.auth_token = auth_token
self.refresh_token = refresh_token
self.headers['Authorization'] = 'Bearer ' + auth_token
def auth_method(self):
if self.qr_code:
payload = {
'password': self.password,
'username': self.username,
'grant_type': 'password',
'client_id': "c82SH0WZOsabOXGP2sxqcj34FxkvfnWRZBKlBjFS",
'scope': 'internal',
'device_token': self.device_token,
'mfa_code': self.get_mfa_token(self.qr_code)
}
try:
res = self.session.post(login(), data=payload, timeout=15)
data = res.json()
if 'access_token' in data.keys() and 'refresh_token' in data.keys():
self.auth_token = data['access_token']
self.refresh_token = data['refresh_token']
self.headers['Authorization'] = 'Bearer ' + self.auth_token
return True
except requests.exceptions.HTTPError:
raise RH_exception.LoginFailed()
else:
payload = {
'password': self.password,
'username': self.username,
'grant_type': 'password',
'client_id': "c82SH0WZOsabOXGP2sxqcj34FxkvfnWRZBKlBjFS",
'expires_in': '86400',
'scope': 'internal',
'device_token': self.device_token,
}
try:
res = self.session.post(login(), data=payload, timeout=15)
res.raise_for_status()
data = res.json()
if 'access_token' in data.keys() and 'refresh_token' in data.keys():
self.auth_token = data['access_token']
self.refresh_token = data['refresh_token']
self.headers['Authorization'] = 'Bearer ' + self.auth_token
return True
except requests.exceptions.HTTPError:
raise RH_exception.LoginFailed()
return False
def logout(self):
"""Logout from Robinhood
Returns:
(:obj:`requests.request`) result from logout endpoint
"""
try:
payload = {
'client_id': self.client_id,
'token': self.refresh_token
}
req = self.session.post(logout(), data=payload, timeout=15)
req.raise_for_status()
except requests.exceptions.HTTPError as err_msg:
warnings.warn('Failed to log out ' + repr(err_msg))
self.headers['Authorization'] = None
self.auth_token = None
return req
`
@klepsydra yes, I think we could that.
Here's my current take on option 2, westonplatter/fast_arrow_auth#2.
I've merged #94 as a 1.0.0 release candidate. Code is hard to get exactly right, so expecting that the current approach doesn't hit all needs. Happy to work through details and make adjustments to get something that's solid and useable.
Closing this issue after release 1.0.0 in the wild. Please open another issue if you run into authentication issues.