digitalmethodsinitiative/4cat

Bitchute datasource does not currently work

dale-wahl opened this issue · 1 comments

Currently BitChute datasets fail to complete. It appears to be some issue with authentication and our current method is no longer working. I have explored the csrftoken token and recreating the cookies but cannot seem to effectively recreate the request to get the proper JSON result.

Failed BitChute request to https://www.bitchute.com/api/search/list/ 3 times, aborting
Stack trace:
threading.py:930 → threading.py:973 → worker.py:117 → processor.py:189 → search.py:84 → search.py:255 → search_bitchute.py:279 → search_bitchute.py:579

Possible methods forward:

  • we rewrite the datasource
    • try to scrape the results from a different request if no one can figure out how to recreate that particular request
    • could use a different scraper (e.g., merge tracker-tracker branch)
  • we deprecate bitchute as a datasource 😢

In the meantime, I have deactivated the datasource from our 4CAT instances.

I tried rewriting it to not trigger the token check and failed, maybe it would work when using Selenium, but I'm not sure that's worth it...