Twitter Sentiment Analysis Using ELK Stack and Python
Workflow
Technical Background
ELK Stack
Python Integration
There are several pacakges in Python that are very useful to this project, see the image below.
Prerequisites
- Install and setup ELK Stack
- Elasticsearch
- Kibana
- Python packages
- Tweepy
pip install tweepy - TextBlob
pip install -U textblob python -m textblob.download_corpora - Elasticsearch-py
pip install elasticsearch
- Tweepy
- Set up Twitter streaming API
- Set up Google Maps API
Usage
Configure credentials
In config.py, type in your own API keys and tokens.
consumer_key = "<Your_Twitter_Consumer_Key>"
consumer_secret = "<Your_Twitter_Consumer_Secret>"
access_token = "<Your_Twitter_Access_Token>"
access_token_secret = "<Your_Twitter_Access_Token_Secret>"
google_api_key = "<Your_Google_Map_API_Key>"Change the names of index and doc_type
If you are not happy with the default values of index and doc_type, you can change them at main.py, line 27-28.
index = "tweet-sentiment"
doc_type = "new-tweet"Run the program with your topics of choice
Run the following command in your terminal
python3 main.py <any-topics-that-you-are-interested-in>
For example
python3 main.py coldplay muse suede
You can also directly run
python3 main.py
It will filter tweets on the default topics #interstellar, #inception, #dunkirk, interstellar, inception, dunkirk.
Sample Output
As the program starts running, you will see ouputs on your console similar to this (the Topics, Index and doc type values might vary depending on your configuration):
==> Topics ['#interstellar', '#inception', '#dunkirk', 'interstellar', 'inception', 'dunkirk']
==> Index: tweet-sentiment, doc type: new-tweet
==> Start retrieving tweets...
=> Retrievd a tweet
[sentiment] positive
[hashtags] Dunkirk
[country] None
[time] 2018-09-29T21:04:01+00:00
=> Retrievd a tweet
[sentiment] neutral
[hashtags] None
[country] None
[time] 2018-09-29T21:04:18+00:00
...
You can also check the streaming status by opening Kibana and clicking on the Discover tab.
