[Help Needed] wait for site finish loading
huynguyen230892 opened this issue · 4 comments
Hello, I'm really sorry to bother you, but how can I get the contents of a website after it has finished loading? I've searched everywhere and found nothing. Please help me. :(
I, too, would like this functionality.
In other headless browsers I have used, there is an option to wait for the page to completely load before scraping or manipulating, including the page's JavaScript updating more "dynamic" pages before querying for information.
A proper signal that says "the page is actually fully loaded and ready to parse" would be really helpful. A lot of web sites use code to immediately load in data, but it's not immediate.
I think other headless browsers do it by detecting when DOM changes really stop after X threshold of time, or maybe detecting that no immediate JavaScript is running, I'm not really sure. But this would be a fantastic feature.
I, too, would like this functionality.
In other headless browsers I have used, there is an option to wait for the page to completely load before scraping or manipulating, including the page's JavaScript updating more "dynamic" pages before querying for information.
A proper signal that says "the page is actually fully loaded and ready to parse" would be really helpful. A lot of web sites use code to immediately load in data, but it's not immediate.
I think other headless browsers do it by detecting when DOM changes really stop after X threshold of time, or maybe detecting that no immediate JavaScript is running, I'm not really sure. But this would be a fantastic feature.
Could be implemented here if api allow it
https://github.com/phimage/Erik/blob/92c7081ce6f5b4416cc238791e14960bc44fdbf7/Sources/LayoutEngine.swift#L65C17-L65C34
my way to do it is after loading wait for a specific dom element created, or some js variable intialized, and if not I want and retry to get it
So it's depend on each page. I do it a lot in browser "userscript"
Hello, I am also receiving different responses, because page have JS and it is not fully loaded. So could you give me a clue how to use this library? How to wait for page is loaded? Thank you