This project consists of two microservices, one for the backend built in python and one for the frontend built in React.
OVERVIEW BACKEND
A wrapper around Praw to fetch comments and posts from UIUC subreddit. This wrapper fetches posts and comments between configurable dates and calculates the sentiment value and stores the data in a sqllite3 database called data.db. The data inside data.db consists of sentiment and postscounts between 2010 up to 2018. To fetch more data one must create a .env file following the format:
{
'CLIENT_ID': 'client_id’,
'CLIENT_SECRET': 'secret_token',
'PASSWORD': ‘password’,
'USERAGENT': app_agent,
'USERNAME': 'reddit username’
}
and register the application at reddit api
This file calculates the sentiments for each comment. This is done by using pythons NLTK library together with the VADER (Valence Aware Dictionary and sEntiment Reasoner) lexicon. Instead of calculating each comment sentiment separate, each comment is weighted by
# num comments in post / # total comments that day * sentiment_for_comment
To get the total sentiment for one day, the sum over all sentiment is used for that particular day.
The server provides the backend REST api for the frontend to be able to fetch data for givens dates. This is done with flask
OVERVIEW FRONTEND
The frontend is built using React and is using React chart js 2 for the graph and airbnb datepicker for choosing dates.
In order to run the program on your local computer you need to clone both projects from github and start them in separate terminals.
git clone git@github.com:erichall/UIUCSentimentBackend.git
cd UIUCSentimentBackend/
pip install requirements.txt
python server.py
OR python server.py --host <HOST, default=localhost> --port <PORT, default=3001> to specify manual host and port.
test it with opening a browser and navigate to
http://localhost:PORT/sentiment_range?start_date=2017-01-01&end_date=2017-02-01
where port is either 3001 or what you specified in the argument above.
You should see a JSON dump that looks something like this:
{"2017-01-01": {"sentiment": 0.5670441860465116, "post_count": 43}, "2017-01-02": {"sentiment": 0.5536891304347826, "post_count": 46}...
You can have navigated to the wrong address, make sure it’s http://HOST:PORT/sentiment_range?start_date=2017-01-01&end_date=2017-02-01 . Yes you need to sentiment_range endpoint with start and end-date.
git clone git@github.com:erichall/UIUCSentimentFrontend.git
cd UIUCSentimentFrontend/
npm install
REACT_APP_SERVER_HOST=server_host REACT_APP_SERVER_PORT=server_port npm start
where server_host = the host you started the server on, default localhost server_port = the port you started the server on, default 3001
navigate to http://localhost:3000/ and your browser and you should see a graf with that shows the sentiment over time.
The most common problem I encountered is browser caching, so either open the frontend page in incognito mode or clear you browser history.
Eric Hallström hallstrom.eric@gmail.com