It doesn't pass robot protection on some websites
brootle opened this issue · 2 comments
Describe the bug
It doesn't pass robot protection on some websites. Here is a good example https://www.bloomberg.com/news/articles/2022-05-17/elon-musk-says-twitter-must-prove-bot-claims-for-deal-to-proceed
Here is what I am getting in response, see "Bloomberg - Are you a robot?"
{ "data": { "url": "https://www.bloomberg.com/tosv2.html?vid=&uuid=a0651c3d-d82c-11ec-bb03-4a7a46654c6c&url=L25ld3MvYXJ0aWNsZXMvMjAyMi0wNS0xNy9lbG9uLW11c2stc2F5cy10d2l0dGVyLW11c3QtcHJvdmUtYm90LWNsYWltcy1mb3ItZGVhbC10by1wcm9jZWVk", "title": "Bloomberg - Are you a robot?", "mediaType": "website", "contentType": "text/html", "images": [], "videos": [], "favicons": [ "https://www.bloomberg.com/favicon.ico" ] } }
To Reproduce
Steps to reproduce the behavior:
Just pass this url https://www.bloomberg.com/news/articles/2022-05-17/elon-musk-says-twitter-must-prove-bot-claims-for-deal-to-proceed to getLinkPreview
function
Expected behavior
Expecting to get same results as Twitter, Facebook or Google news gets
Facebook, Google, and Twitter are probably trusted domains by Bloomberg. Nothing this library can do about it.
yeah, I think that's the case, thanks!