Project uses ML techniques in detecting a probable suicide message based on social media posts. For this purpose, we trained and tested classical machine learning classifiers such as Naïve Bayes, Support Vector Model, and Logistic Regression distinguish Reddit posts that indicate suicide and non-suicide. The word associations derived from each method is used to identify posts with suicidal tendencies.
| Datasets | Intended Use | Rows | Description |
|---|---|---|---|
| Suicide Detection | Training | 232,074 | Data from Kaggle. Data contains reddit posts that have been labelled as suicide and non-suicide. |
| Suicide Notes | Test | 464 | Data from Kaggle. Notes written by users who were confirmed with suicidal tendencies. |
| r/depression r/SuicideWatch | Test | 20,364 | Data from Kaggle. Data contains reddit posts from r/depression and r/suicidewatch. |
Dataset is balanced and accurately labelled.
| Model | BERT Accuracy | Count Vectorize Accuracy |
|---|---|---|
| Logistic Regression | 81.08 | 93.19 |
| Bernoulli Naive Bayes | 82.33 | 77.75 |
| Multinomial Naive Bayes | 82.33 | 90.25 |
| SVM | 82.33 | 92.55 |
| MLP | 82.07 | 94.52 |
Dataset is unbalanced and skewed to contain only sentences labelled as 'suicide'.
| Model | BERT Accuracy | Count Vectorize Accuracy |
|---|---|---|
| Logistic Regression | 59.50 | 83.41 |
| Bernoulli Naive Bayes | 52.86 | 59.05 |
| Multinomial Naive Bayes | 36.16 | 94.83 |
| SVM | 59.04 | 78.66 |
| MLP | 60.41 | 97.84 |
Dataset is balanced, but labels need not be accurate.
| Model | BERT Accuracy | Count Vectorize Accuracy |
|---|---|---|
| Logistic Regression | 44.98 | 54.61 |
| Bernoulli Naive Bayes | 45.02 | 45.30 |
| Multinomial Naive Bayes | 45.45 | 49.43 |
| SVM | 44.86 | 54.58 |
| MLP | 45.89 | 53.73 |
Count Vectorize perfromed better than BERT as a tokenizer. Multinomial Naive Bayes classifier performed the best.
| Datasets | True Positives | False Positives | False Negatives |
|---|---|---|---|
| Testset | regret | social anxiety | loneliness & depression |
| Suicide Notes | goodbye notes | loneliness & anxiety | |
| r/depression r/SuicideWatch | regret & goodbye notes | social anxiety | loneliness & anxiety |
