/stackoverflow-sentiment

Uses BigQuery & Natural Language API to analyse Stackoverflow comment sentiment

Primary LanguagePythonApache License 2.0Apache-2.0

Stack Overflow Sentiment

This simple project uses Google BigQuery & Google Cloud Natural Language API to analyse Stack Overflow comment sentiment. Developed using Python 2.7.x (but should work with Python 3.x though too). The queries and datasets used are all public. You can find the links below.

##Dependencies There are some Python modules needed. Simply run this. pip install -r requirements.txt

##To run:

python sentiment/stackoverflow_sentiment.py <path_to_service_account_json_file> <dataset>

##Links

###Blog post https://shinesolutions.com/2016/12/20/analysing-stack-overflow-comment-sentiment-using-google-cloud-platform/

###Google Tools https://cloud.google.com/natural-language/ https://cloud.google.com/bigquery/

###Google APIs https://github.com/GoogleCloudPlatform/google-cloud-python/tree/master/bigquery https://github.com/GoogleCloudPlatform/google-cloud-python/tree/master/language

###Motivation https://cloud.google.com/blog/big-data/2016/12/google-bigquery-public-datasets-now-include-stack-overflow-q-a

##Public stuff

###Datasets

####Stack Overflow Public Dataset (BigQuery) Here's the public dataset in BigQuery for Stackoverflow https://bigquery.cloud.google.com/dataset/bigquery-public-data:stackoverflow

####My Public Dataset (BigQuery) Here you can find the tables that I created with my queries and the results of the NL-API https://bigquery.cloud.google.com/dataset/stackoverflow-sentiment:sentiment

###Queries

####Best day to post a question on Stack Overflow https://bigquery.cloud.google.com:443/savedquery/1056242102418:63c31fe61a7c4d53ae3ef22f30203881

####Correctly spelled BigQuery - it's one word! :-) https://bigquery.cloud.google.com:443/savedquery/1056242102418:b4117babd35c41e78aa0ab8c69f42257

####Overall % sentiment https://bigquery.cloud.google.com:443/savedquery/1056242102418:864b7503d0d3407bb97b08350fbdc311

####Overall sentiment counts https://bigquery.cloud.google.com:443/savedquery/1056242102418:9a7847826552407aa690db60a1276feb

####Average sentiment https://bigquery.cloud.google.com:443/savedquery/1056242102418:7b4dccb9c54b45ed86c4876df5e4794a

####Comments on questions asked by noobs for 2016 and for JavaScript, Java and Python tags https://bigquery.cloud.google.com:443/savedquery/1056242102418:1c6c51620ee24e878cb361e0e236426b

####Counts of negative comments https://bigquery.cloud.google.com:443/savedquery/1056242102418:57b1910592874c3fa4db79a103797cd9