An application using Twitter streaming API to generate reports about different aspects.
You should have the following packages installed system-wide
- Python 3
- Pip
- Virtualenv
- Redis
In order to initialize the project, follow the commands written below in the exact sequence:
- git clone https://github.com/shivan1b/twireport.git
- virtualenv venv
- source venv/bin/activate
- pip install -r requirements.txt
Now, your project should have been initialized with all the requirements.
Making use of Twitter's API will require credentials for authenticating the app for using the API. Get your authentication credentials here: https://apps.twitter.com/ after giving your app a fancy new name. Following this, create a file named .env in the root of the project and add the following variables and their corresponding values that you received from Twitter:
ACCESS_TOKEN='my_access_token'
ACCESS_SECRET='my_access_secret'
CONSUMER_KEY='my_consumer_key'
CONSUMER_SECRET='my_consumer_secret'
This project makes use of nltk
python package which requires download of dataset in the following manner.
$ ▶ ipython
Python 3.6.5 (default, May 11 2018, 04:00:52)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import nltk
In [2]: nltk.download()
In order to run the program, get into the root directory of the project and type,
$ ▶ python pilot.py
You shall now be prompted for the keyword you'd like to stream Twitter for
Please enter the keyword which you'd like to track:
linux
After typing the keyword, press enter and open up another terminal to type
$ ▶ python stream.py
and wait for Twireport
to generate beautiful reports for you.
Twireport
shall generate a report similar to below on the console where you ran stream.py
User Report
********************************************************************************
+-----------------------------------------------------------+------------------+
| Username | Number of tweets |
+-----------------------------------------------------------+------------------+
| karem cargo a alan. | 1 |
+-----------------------------------------------------------+------------------+
| Majo Fon | 1 |
+-----------------------------------------------------------+------------------+
| Cléo | 1 |
+-----------------------------------------------------------+------------------+
| conch shell 🐚 pls commission me i need 7000€ | 1 |
+-----------------------------------------------------------+------------------+
| Appauth | 3 |
+-----------------------------------------------------------+------------------+
|
Link Report
********************************************************************************
Link count: 6
+----------------+
| Unique Domains |
+----------------+
| google.com |
+----------------+
| twitter.com |
+----------------+
| mozilla.org |
+----------------+
| curiouscat.me |
+----------------+
Word Report
********************************************************************************
Unique word count: 10
+----------------+
| Content Report |
+----------------+
| asdfghjkl |
+----------------+
|
Following are the improvements that I see can be made.
pilot.py
could be run as a daemon. Then,stream
could be used as a command.- Performance improvements for very large datasets.
If you find any issue with the current implementation or have a more optimized solution, please consider sending a Pull Request. Make sure to have a different and descriptive name for your branch.