The final code for masters thesis for BCU Big Data Analytics. Scrape twitter for Brexit tweets, then use Transfer Learning technique to evaluate tweets for signs of depression and anxiety.
As of the time of this writing, Brexit is on the tip on the tongue for most of us living in the United Kingdom. From the onset of the Brexit referendum, the decision for the United Kingdom (UK) to leave the European Union (EU) was politically and socially polarizing. In the years since the decision, the uncertainty around the UK’s continued relationship with the EU, and its impact on the UK after the “divorce”, has only increased. While the potential consequences are often spoken of in terms of socio-economic impact, for example the food-shortages and unemployment predicted by “Operation Yellowhammer”, there has also been an interest in understanding the potential psychological impact. This research compared machine learning models in their ability to automatically identify signals of psychological distress for Twitter users with self-identified anxiety and depression. Transfer learning and domain adaptation techniques were then applied to estimate signals of psychological distress in #Brexit Tweets. Compared to similar research identifying mental health signals, this research achieved moderate classification accuracy. Importantly, when applied to a large sample of Brexit Tweets from August and September 2019, the Brexit Distress Index identified signals of distress which correlated with major Brexit events such as the surprise parliament prorogation and the release of “Operation Yellowhammer”. Distress-indicative Tweets were identified for those who were pro-remain as well as for those who were pro- leave. These results provide quantifiable evidence that Brexit may have a wide- reaching negative impact on the psychological well-being of UK citizens. This research presents an effective approach to automatically monitor changes in psychological well-being for UK citizens, and it only requires data which is readily accessible from the Twitter API. As such, the Brexit Distress Index may be useful for continued monitoring of mental health needs as the UK proceeds through its historic change.
