The script tweets_sentiments.py uses the MapReduce distributed computing paradigm to perform sentiment analysis using tweets collected through the Twitter API. Specifically, it utilizes MRJob, which allows simulating the behavior of a distributed system locally, as well as running on real distributed environments (such as Cloudera) and with the AWS EMR service.
The goal of this repository is to provide a service that retrieves tweets using tweepy, analyze the sentiment of each tweet by looking each word in a precomputed sentiment analysis dictionary and aggregate the sentiment for each Spanish administrative locality.
It uses 2 files:
- Redondo_words.txt: contains a list of words together with an associated sentiment (0 negative, 10 positive)
- comunidades.json: JSON file contains Spanish administrative localities together with a Shapely polygon that limits each area.
It has 3 operating modes, specified by the --job-options parameter in the execution command:
- sentiments: calculates the aggregated sentiments of each community/province in Spain.
- most-happy: calculates the happiest community/province based on previous sentiment scores.
- trending: prints the top 10 trending topics that are most frequently mentioned.
python tweets_sentiments.py -r local tweets.json --file Redondo_words.txt --file comunidades.json --job-options sentiments
python tweets_sentiments.py -r emr s3://bucket/tweets.json --file s3://bucket/Redondo_words.txt --file s3://bucket/comunidades.json --output-dir=s3://bucket/output --job-options sentiments