IonicaBizau/scrape-it

API return JSON with HTML property

AleTid5 opened this issue · 2 comments

AleTid5 commented

Hi there!
I am using this library and I realized that there is a case that is not contemplated, it seems to me.
The API returns the following JSON:
{"status":"success","content":{"block":"<div class=\"category-products\">\r\n <div class=\"toolbar\">\r\n...
Any idea how to scrap it? (My intention is not to use an external library)

Regards!

IonicaBizau commented

No, you will have to parse the response manually and send it to scrapeIt.scrapeHTML(parsedData.content.block).

AleTid5 commented

Yup! I solved it using this piece of code (may be I help someone):

const scrapAjax = (page) => {
  scrapeIt(
    "http://url/to/api",
    null,
    async (err, res) => {
      const {
        content: { block, last },
      } = JSON.parse(res.body); // JSON.parse(res.body) = {status: true, content: {block: "<div><p>...", last: false|true}}

      const products = scrapeIt.scrapeHTML(block, { data }); // data = {listItem: "div...", data: {...}}

      if (!last) {
        // ToDo: Wait 1 second until next call.
        scrapAjax(page + 1);
      }
    }
  )
};

Thanks!! 🚀

  • 👍1