clemfromspace/scrapy-selenium

Integration with 'selenium-stealth'

afapa1-iscteiulpt opened this issue · 2 comments

Hello,

Is there a way to use it with https://pypi.org/project/selenium-stealth/ ?

I tried to use it in the parse function

def parse_realEstate(self, response):

    driver = response.meta['driver']
    
    stealth(driver,
            languages=["en-US", "en"],
            vendor="Google Inc.",
            platform="Win32",
            webgl_vendor="Intel Inc.",
            renderer="Intel Iris OpenGL Engine",
            fix_hairline=True,
            )

but it doesn't work.
I tried with the script argument but without success.

yield SeleniumRequest(url=realEstate, callback=self.parse_realEstate, wait_time=60, wait_until=EC.visibility_of_element_located ((By.XPATH, '//*[@id="__next"]/main/div[*]/div[*]/header/strong')), script="...")

Any suggestion to make it work?

Thanks.

Hi! Purely personal opinion... Maybe try to use stealh(*args, **kwargs) when initializing middleware? I had a similar experience with playwright-stealth and there it was enough to create a custom middleware and in it add necessary methods to the driver like this.

As I am no expert with Python, the solution I opted for was to grab clone the repository and added the code needed for the library 'selenium-stealh' after the creation of the driver. This repo is not being updated so this way if I need a future new improvement I can update it myself. Thanks for the suggestion anyway.