401 Client Error: Unauthorized for url

[nathanramoscfa]

I am trying to automate login and use the resulting session object with pyetrade modules. However, I am getting 401 Client Error when trying to get account balance, list accounts, list orders, or preview equity order with pyetrade modules. However, for some functions, I am able to get a response if I use xmltodict.parse({login.session}.get(LIVE URL).text) on the login object. Here is the automated login function:

from rauth import OAuth1Service
import undetected_chromedriver.v2 as uc
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
import time
import xmltodict
import json
import random
import pyetrade
import urllib.parse

class AutomatedLogin():
    def __init__(self, consumer_key, consumer_secret, web_username, web_password):
        self.consumer_key = consumer_key
        self.consumer_secret = consumer_secret
        self.web_username = web_username
        self.web_password = web_password
        self.service = OAuth1Service(
                  name='etrade',
                  consumer_key=consumer_key,
                  consumer_secret=consumer_secret,
                  request_token_url='https://api.etrade.com/oauth/request_token',
                  access_token_url='https://api.etrade.com/oauth/access_token',
                  authorize_url='https://us.etrade.com/e/t/etws/authorize?key={}&token={}',
                  base_url='https://api.etrade.com')
        self.oauth_token, self.oauth_token_secret = self.service.get_request_token(params={'oauth_callback': 'oob', 'format': 'json'})
        self.auth_url = self.service.authorize_url.format(consumer_key, self.oauth_token)
        self.verifier = self.__get_verifier(headless=True, action_delay_sec=2)
        if self.verifier:
            self.session = self.service.get_auth_session(self.oauth_token, self.oauth_token_secret, params={'oauth_verifier': self.verifier})

    def __get_verifier(self, headless=True, action_delay_sec=2):
        if headless:
            options = uc.ChromeOptions()
            options.headless = True
            options.add_argument('--headless')
            driver = uc.Chrome(options=options, driver_executable_path='H:\My Drive\etradebot\chromedriver.exe')
            print(f'Headless web login with action_delay set to: {action_delay_sec} seconds')
        else:
            driver = uc.Chrome()
        web_action = ActionChains(driver)

        try:
            with driver:
                driver.get(self.auth_url)
                # log in
                username = driver.find_element(By.NAME, "USER")
                password = driver.find_element(By.NAME, "PASSWORD")
                username.send_keys(web_username)
                password.send_keys(web_password)
                driver.find_element(By.ID, "logon_button").click()
                time.sleep(action_delay_sec)
                web_action.send_keys(Keys.TAB).send_keys(Keys.RETURN).perform()
                return driver.find_element(By.TAG_NAME, "input").get_attribute("value")
        except Exception as e:
            print(str(e))
            return

# Automated Login
login = AutomatedLogin(consumer_key, consumer_secret, web_username, web_password)

I then obtain tokens like this:

tokens = {
    'oauth_token': login.oauth_token,
    'oauth_token_secret': login.oauth_token_secret
}

which looks like this (not real tokens, just demonstrating):

tokens
{
    'oauth_token': 'FfegWsfgdhfrhfew94hf84h399hfer/gfdgRgrfd=',
    'oauth_token_secret': 'greFGnfeFESFsFSfsfsfsfhgedgsssfsfdfGFRSsDbkjju='
}

If I try to get accounts list with pyetrade like this:

accounts = pyetrade.ETradeAccounts(
    consumer_key,
    consumer_secret,
    tokens['oauth_token'],
    tokens['oauth_token_secret'],
    dev
)

print(accounts.list_accounts(resp_format='xml'))

results in:

HTTPError: 401 Client Error: Unauthorized for url: https://api.etrade.com/v1/accounts/list

But when I use this code instead, it works:

xmltodict.parse(login.session.get('https://api.etrade.com/v1/accounts/list', params={'format': 'json'}).text)

If I try to get account balance, neither method works. For example, if I try with pyetrade:

print(accounts.get_account_balance(accountIDKey, resp_format='xml'))

I get (redacted):

HTTPError: 401 Client Error: Unauthorized for url: https://api.etrade.com/v1/accounts/{accountIDKey}/balance?
realTimeNAV=True&instType=BROKERAGE

If I try the other method to get account balance:

xmltodict.parse(login.session.get('https://api.etrade.com/v1/accounts/{}/balance?instType=
{}&realTimeNAV=true'.format(accountIDKey, 'BROKERAGE')).text)

I get:

{'Error': {'message': 'oauth_problem=signature_invalid'}}

In my google search, I found someone saying (https://stackoverflow.com/a/66405709/19349500), "The reason turned out to be that the oauth_signature and other parameters must be percent-encoded (rfc3986)." Could this be the reason to my problem? I tried to convert my tokens to rfc3986 percent-encoded like this:

tokens_rfc3986 = {
    'oauth_token': urllib.parse.quote(login.oauth_token),
    'oauth_token_secret': urllib.parse.quote(login.oauth_token_secret)
}

Convert tokens to look like this

tokens
{
    'oauth_token': 'FfegWsfgdhfrhfew94hf84h399hfer%gfdgRgrfd%3D',
    'oauth_token_secret': 'greFGnfeFESFsFSfsfsfsfhgedgsssfsfdfGFRSsDbkjju%3D'
}

But to no avail. Thanks for any help.

[mw66]

Try this, maybe you don't have to re-invent the wheel:

#65

[jessecooper]

Please move this to the discussion here: #64