nyavramov/monitor_web_page_changes

Can this auto-log in to a website account, and then monitor from there?

Opened this issue · 4 comments

KeronCyst commented

The page I'm trying to track for monitoring is hidden behind a mandatory login.

nyavramov commented

Hey KeronCyst,

I'd need to change the code slightly to allow it to log in to a specific web page. Selenium, the package I'm using in this library, can easily be used to log in to web pages after opening a link.

If you're okay with sharing the website you're trying to log in to, I can update my script allow you to pass your credentials in. If not, I suggest the following:

  • Open the page you're trying to log in to
  • Locate the username and password HTML elements using chrome developer tools
  • Use Selenium to locate and then fill in the text fields of those HTML elements
  • Use Selenium to press enter or click on login button after you've filled in the credentials

This should work if there's no captcha verification at the login screen. If there is a captcha, things become a whole lot more complicated, though not impossible.

Let me know if you need further help with this.

KeronCyst commented

Thank you so much for your swift response! The website is non-CAPTCHA: https://adminweb.aesoponline.com/access

I was reading https://chrisalbon.com/python/web_scraping/monitor_a_website/ but it doesn't have any login section.

nyavramov commented

No problem. Add this code to line 59 of monitor.py and it should log you in:

time.sleep(3)
        
username_element = self.driver.find_element_by_id("Username")
password_element = self.driver.find_element_by_id("Password")
login_button_element = self.driver.find_element_by_id("qa-button-login")

username_element.send_keys("YOUR_USERNAME_HERE")
password_element.send_keys("YOUR_PASSWORD_HERE")
        
login_button_element.click()
KeronCyst commented

Unfortunately, it looks like this particular page seems to go through multiple URLs before finally landing at the one with the actual data that I want to track, and I can't tell if the script is getting there/doing anything after having run it with the addition above, but I appreciate the effort... I should probably mention that I'm a super-newbie to programming languages so figuring this out is probably beyond my league. Thanks anyways, though.