the extracted content output not contain picture elements
wizos opened this issue · 8 comments
Please include more details, including:
- A specific URL that demonstrates the problem.
- The exact markup downloaded from that URL by a JavaScript-disabled User Agent, e.g.
wgetorcurl. - What you expected to see.
- What you actually saw.
I think they just mean that the extracted text doesn't have the original images within the text - is there a way to do this?
<img> elements should be included in the output DOM, so if they're not, then it needs to be debugged. Different sites have different markup, so it's hard to debug without a test case.
OP hasn't replied in a long time, but if you have an example URL + markup, please attach it here.
@chimbori is this being worked on? Still not getting any tags
Not being actively worked on, no. I’ll look into it if/when I have a chance, but the reason I asked for more documentation is that others who see this issue could have enough information to get started.
If anyone does look at this, the reason it doesn't work is because some sites load some of their images lazily with JavaScript, and the HTML you are providing is likely the one before the images are inserted. To fix this, the JavaScript must first be run, then provide that HTML to Crux - this can be done with something like HtmlUnit, but that library doesn't work on Android.
Still trying to find a solution to that, though that might be out of the scope of Crux - with the post-JavaScript HTML, it works fine.
Am now working on this - by the way, where is the code that should make the elements included in the output DOM? I couldn't find any, had to add my own to get any working - when I pass all the tests, I'll submit a merge request.