OP-Engineering/link-preview-js

Attention Required! | Cloudflare

wouterlemcke opened this issue · 3 comments

wouterlemcke commented

Some url's successfully return a link preview object. However the title is Attention Required! | Cloudflare, which we don't see when we open the page in the browser.

As far as I can see there are no redirect or anything on the page I'm trying to load. Is there something I can do to fix this?

An example url is:
https://bible.com/bible/1990/rom.8.31.HSV

ospfranco commented

Cloudfare bot detection has probably marked your IP address or somehow detected a bot and is blocking your requests. You can try to play with the user agent as some of the big corpos are whitelisted (Apple uses facebook's and twitter's). Nothing wrong with the library, I would suggest asking on Stack Overflow, some ppl might have some ideas.

ospfranco commented

Ah sorry, I misread you do get a response, but with the wrong title. No idea, each page can return w/e they like, this just reads the tags.

sergei-zelinsky commented

For anyone experiencing this issue in the future. I did overcome it using the Slackbot user agent header.

For example:

const preview = await getLinkPreview(url, {
  followRedirects: 'follow',
  headers: {
    'user-agent':
      'Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)',
  },
});
  • ❤️1