The Python scripts and IPython notebooks in this repo collect, clean, preprocess and perform topic modeling on all Tweets of the current members of the US Congress. They also introduce campaign finance information for each member of Congress and investigate the relationship between the amount of PAC (Political Action Committee) members receive to finance their campaigns and what they Tweet about. In addition, this map shows the amount of money PACs contributed to the campaigns of incumbent members of the U.S. House of Representatives in 2018.
-
get_congress_tweets.py: gets the Tweets of all the current members of the US Congress, and writes the Tweets and select Tweet metadata as CSV files
-
partial_data_eda.ipynb: reads in Senator Amy Klobuchar's Tweets performs EDA on the them
-
tweets_to_pickle.ipynb: reads all Tweets from CSV files and then pickles the data
-
full_data_clean_preprocess.ipynb: reads all Tweets from pickled file, performs EDA, cleans and preprocesses the Tweets in preparation for topic modeling
-
congress_metadata.ipynb: downloads metadata for each Congress member, and resolves discrepancies between the Twitter usernames listed in the Congress metadata and those listed alongside the Tweet data
-
topic_modeling.py: reads preprocessed Tweets from pickled file, converts preprocessed Tweets to a matrix of token counts, and performs Latent Dirichlet Allocation to derive topics from and assign topics to the Tweets
-
campaign_finance_data.ipynb: downloads campaign finance information for each relevant member of Congress
-
campaign_finance_eda_aggregate_pac.ipynb: performs EDA on aggregate campaign finance data for current Congress members, and aggregates PAC contribution totals for each relevant member of Congress
-
topic_pac_analysis.ipynb: shows the amount of PAC money received by Congress Members by state, party, etc., abbreviates topics for analysis, shows the top Tweet topics by Congress Members who received the most and least money from PACs to finance their most recent campaigns, and explores the high level differences between in PAC contributions depending on what Congress Members Tweet about
-
Map.md: shows the amount of money PACs contributed to the campaigns of incumbent members of the U.S. House of Representatives in 2018
-
Presentation.pdf: presentation of process and results