justinlittman/twitter-scraper

Scrape restricted twitter accounts

DonaldTsang opened this issue · 4 comments

DonaldTsang commented

Can this scraper scrape tweets like https://twitter.com/jackiec57 ?

To remove the need to click "show sensitive content anyways", it would need to...

  1. Login just like https://gist.github.com/momota10/969d904b4cad239da2a5c00df1ad87e7
  2. Go to https://twitter.com/settings/safety
  3. Check the "show sensitive" box in
    <input name="search-settings-nsfw" checked="" class="SearchSettings-toggleInput SearchSettings-toggleInput--sensitive" type="checkbox">
  4. click "submit" in <button id="settings_save" class="EdgeButton EdgeButton--primary EdgeButton--medium" type="submit" disabled="">Save changes</button>
justinlittman commented

It isn't designed to scrape accounts with sensitive contents.

DonaldTsang commented

@justinlittman but it can be modified to do so, as it uses Selenium. All it has to do is login, no?

justinlittman commented

Yes, I believe so. However, I don't believe that you can hydrate protected tweets from Twitter's API (though I could be mistaken).

igorbrigadir commented

I think you can hydrate protected tweets from API if your account follows the protected account (but i'm also not sure about this and need to double check in case this changed).

Protected tweets don't appear at all in searches though, so you can't display from:protected_account.

It is possible to "login" and get tweets that may be hidden as "sensitive" - I have that half working in my fork.