Twitter Discovery lets you search articles referenced from your tweets, retweets, and favorites.
Install Twitter Discovery from source using pip install -e ..
(And do yourself a favor and use venv.)
Also install twitter-to-sqlite using pip install
twitter-to-sqlite. This is the tool that you'll use to download
data from Twitter. Follow the instructions in the twitter-to-sqlite
docs and set up authentication with Twitter.
Note that dnouri is my own Twitter handle, so whenever you encounter that in the commands below, you should replace dnouri with your own Twitter handle.
Here's how you first download data from Twitter:
- Retrieve your tweets and retweets using
twitter-to-sqlite user-timeline dnouri.db. - Add favorites using
twitter-to-sqlite favorites dnouri.db.
Now you're ready to find in the tweets that you downloaded any links
to news articles and blogs. The following command will find all such
URLs and it will attempt to download the linked documents and save
them for further use: twitter-discovery articles dnouri.db dnouri.
Finally, use the web-based GUI to search all those articles. To start
it up, run: streamlit run twidis/main.py gui
articles-dnouri.csv.gz.
The GUI allows you to search articles by text, order them by date or
website, and so on. The text search is case insensitive and supports
the use of quotation marks to group words, such that a search for
"big finance" will return articles that contain the phrase big
finance, while searching for big finance without the quotes will
return articles that contain both the word big and the word
finance. The text search will match text inside the article's body,
but also in the article's title and the text of the tweet that
referenced the article.