Title: Discussion Quality Measurement on Social Media: Developing and Validating Dictionaries Based on an Open Vocabulary Approach
Abstract:
Social media data creates an influx of data, where traditional methods for examining public opinion and discourse quality can no longer reasonably scale for theoretical and thematic insights from millions of participants. This study examines whether text classifiers to measure the quality of political talk can generalize to new datasets. First, six classifiers were developed following an open-vocabulary approach based on an annotated mixed social media dataset. Next, through performance evaluations against four other hand-annotated datasets from previous work, the models show modest generalizability at measuring the quality of political talk in other social media platforms. Finally, the study concludes by summarizing the strengths and weaknesses of applying machine learning methods to social media posts and theoretical insights about the quality and structure of online political discourse.
Links:
Paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3870554
Github Repository: https://github.com/kj2013/twitter-deliberative-politics