Login Taobao use transfer_driver_cookies_to_session, but the session is invalid
qilongqiu opened this issue · 5 comments
How should I do
Hey @limpair do you mind providing more detail for the issue? Maybe the cookies that are causing the invalid session?
def login():
s = Session(webdriver_path='chromedriver',
browser='chrome',
default_timeout=15,
webdriver_options={'arguments': ['headless']})
'''use requestium'''
# s.driver.get(Taobao.LOGIN)
# s.driver.find_element_by_class_name('login-switch').click()
# time.sleep(1)
# s.driver.find_element_by_id('TPL_username_1').send_keys('***')
# time.sleep(1)
# s.driver.find_element_by_id('TPL_password_1').send_keys('***')
# time.sleep(1)
# s.driver.find_element_by_id('J_SubmitStatic').click()
# s.transfer_driver_cookies_to_session()
'''use selenium'''
driver = webdriver.Chrome()
driver.get(Taobao.LOGIN)
driver.find_element_by_class_name('login-switch').click()
time.sleep(1)
driver.find_element_by_id('TPL_username_1').send_keys('***')
time.sleep(1)
driver.find_element_by_id('TPL_password_1').send_keys('***')
time.sleep(1)
driver.find_element_by_id('J_SubmitStatic').click()
time.sleep(5)
#print s.driver.page_source
'''is ok'''
print driver.find_element_by_xpath('//*[@id="J_SiteNavLogin"]/div[1]/div[2]/a[1]').text
s.transfer_local_driver_cookies_to_session(driver)
'''failed'''
resp=s.get('https://www.taobao.com/')
print resp.xpath('//*[@id="J_SiteNavLogin"]/div[1]/div[2]/a[1]//text()').extract_first()
transfer_local_driver_cookies_to_session like my pull request
Login requestium also used successfully, but the use of transfer_driver_cookies_to_session session is invalid
As the biggest online shopping site in China,taobao.com always has a way to prevent you crawl it by non- webrowser methods.
@limpair I think it is possible that taobao is using a js script that detects if your browser can run js, if it can't then it blocks you out. Some big sites do this as @vc5 said.
I haven't looked at the site though, so this is just speculation. This could still be an actual requestium bug, so I will test the site and get back to you, thanks for the issue report!
Closing as stale.