This is the repository for:
Introducing CAD: the Contextual Abuse Dataset Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, Rebekah Tromble, NAACL 2021 Paper (ACL Anthology)
The data + models have been uploaded to Zenodo: http://doi.org/10.5281/zenodo.4881008
See the environments
folder for the exact environments that were used. The LR experiments were performed on a local machine. The experiments with BERT and DistilBERT were performed on a computing cluster.
- scikit-learn : 0.22.1
- mlflow: 1.11.0
- nlp: 0.4.0
- pandas
- transformers 3.2.0
Directories:
environments
: Environments used for performing the experiments (Conda).experiments
: The models and model outputs (not included on Github, get this on Zenodo)src
: Python codedata
: data (not included on Github, get this on Zenodo)
Note about the dataset (v1 vs. v1.1)
cad_v1
was used to produce the results in the NAACL 2021 paper.
We identified some minor issues later. This affects the primary and secondary categories of 95 entries. The new version CAD v1.1 is also provided, based on the changes recorded in data/errata_v1_to_v1_1
.
Overview
.
├── data
│ ├── cad_v1.tsv
│ ├── cad_v1_train.tsv
│ ├── cad_v1_dev.tsv
│ ├── cad_v1_test.tsv
│ ├── cad_v1_1.tsv
│ ├── cad_v1_1_train.tsv
│ ├── cad_v1_1_dev.tsv
│ ├── cad_v1_1_test.tsv
│ ├── test
│ │ ├── test_data.tsv
│ │ ├── test_data_test.tsv
│ │ ├── test_predictions.tsv
│ ├── errata_v1_to_v1_1
│ ├── cad_codebook_april2021.pdf
├── experiments
│ ├── lr
│ ├── test_dbert
│ │ ├── cad_final_601_DistilBERT_0_0.00005_False
│ │ ├── cad_final_78_DistilBERT_0_0.00005_False
│ │ ├── cad_final_8923_DistilBERT_0_0.00005_False
│ │ ├── cad_final_92064_DistilBERT_0_0.00005_False
│ │ ├── cad_final_971242_DistilBERT_0_0.00005_False
│ │ ├── test_output_label_analysis_dbert_aggregated.txt
│ │ ├── test_output_metrics_dbert_aggregated.txt
│ ├── test_bert
│ │ ├── cad_final_601_BERT_0.01_0.00003_False
│ │ ├── cad_final_78_BERT_0.01_0.00003_False
│ │ ├── cad_final_8923_BERT_0.01_0.00003_False
│ │ ├── cad_final_92064_BERT_0.01_0.00003_False
│ │ ├── cad_final_971242_BERT_0.01_0.00003_False
│ │ ├── test_output_label_analysis_bert_aggregated.txt
│ │ ├── test_output_metrics_bert_aggregated.txt
│ │ ├── bert_error_analysis.txt
│ ├── tune_dbert_summary.txt
│ ├── test_dbert_summary.txt
│ ├── tune_bert_summary.txt
│ ├── test_bert_summary.txt
├── environments
├── src
Data exploration:
data_analysis.ipynb
A notebook that explores the data and prints out statistics.
Writing the splits:
Given the full file, we can generate individual files for each split (containing id, text, labels) using:
python classification_util.py write_splits --input_file="../data/cad_v1.tsv"
The generated files are also provided as ../data/cad_v1_train.tsv
, ../data/cad_v1_dev.tsv
and ../data/cad_v1_test.tsv
Data checks:
python check_data.py --input_file "../data/cad_v1.tsv"
Logistic Regression is implemented using scikit learn (0.22.1).
The code makes use of the mlflow
library.
All results are initially stored in a mlflow
folder (in the current working directory).
The final model and results are provided in experiments/lr
Tune parameters
python lr.py tune
To view results:
Let's assume the results (the mlflow
folder) is stored in experiments/lr/tuning
. Go to this folder and run:
mlflow ui
See results at http://localhost:5000/
Testing
Based on tuning, the following hyperparameters were used: max_df=0.95, min_df=5, C=0.25
python lr.py test
Analysis
Both calls below work in the same way. Based on the file with the predictions for each test instance ( model_outputs.txt
), they will compute additional evaluation statistics and save the results to the same directory of model_outputs.txt
.
Calculate some additional evaluation metrics, such as recall for labels based on previous or current content
python classification_analysis.py general_analysis --predictions_file "../experiments/lr/test/mlruns/0/d0a17039a5104dd8a32b6f8980fe1ace/artifacts/model_outputs.txt"
Do an analysis of the predicted labels (e.g. does the model often underpredict the number of labels?
python classification_analysis.py label_analysis --predictions_file "../experiments/lr/test/mlruns/0/d0a17039a5104dd8a32b6f8980fe1ace/artifacts/model_outputs.txt"
The implementation uses the Transformers library (3.2.0).
The experiments/test_dbert
contains the predictions and analyses files for the five final models (because we used 5 different random seeds).
The folder of the best performing model experiments/test_dbert/cad_final_8923_DistilBERT_0_0.00005_False
also contains the trained model.
Finetuning with different hyperparameters on train and dev
python hatespeech_finetuning_multilabel.py tune DistilBERT [output_dir]
Analyze results
Print out the results (macro and micro F1 scores) for each parameter setting. The method assumes each subdir in the specified directory (replace with the correct path) contains one model:
python analyze_transformers.py analyze_runs --input_pattern "../experiments/tune_dbert/*"
The results are printed to standard output. We have provided the output in experiments/tune_dbert_summary.txt
Best setting: learning rate 5e-5, weight decay 0
Finetune on train with selected hyper parameters, apply model on test
First train and test DistilBERT (5 random seeds) on the train and test set with the selected parameters
python hatespeech_finetuning_multilabel.py test DistilBERT [output_dir]
Each model is stored in a directory. The directory names contain the different parameters used, including the random seed.
Analysis of the results on the set set
- Print out average micro and macro scores for all models who match the specified input pattern (in this case, same settings for DistilBERT runs but with different random seeds).
python analyze_transformers.py analyze_runs --input_pattern "../experiments/test_dbert/cad_final_*_DistilBERT_0_0.00005_False"
The results are provided in experiments/test_dbert_summary.txt
- To do more fine-grained analyses, run model again on test set and print out predictions:
python analyze_transformers.py write_predictions --input_pattern "../experiments/test_dbert/cad_final_*_DistilBERT_0_0.00005_False" --model_type DistilBERT
In each folder, the predictions are stored in a file called test_output.txt
, with the format [postid]\t[multi hot encoding of predictions]
- For each run, read in the predictions file and write out all the metrics:
python analyze_transformers.py print_metrics --input_pattern "../experiments/test_dbert/cad_final_*_DistilBERT_0_0.00005_False" --analysis_type "general_analysis"
This generates two files in each model folder: test_output_analysis.txt
and test_output_metrics.txt
.
Also run this once with analysis_type="label_analysis"
, this will output statistics related to the number of labels predicted, and will generate test_output_label_analysis.txt
For each DistilBERT model, we now have files with evaluation metrics. The following reads in the specified file for each model, and writes out mean and stds for each metric to standard output:
python analyze_transformers.py print_aggr_analysis --input_pattern "../experiments/test_dbert/cad_final_*_DistilBERT_0_0.00005_False" --metrics_file "test_output_metrics.txt"
The output is provided in ../experiments/test_dbert/test_output_metrics_dbert_aggregated.txt
The experiments/test_bert
contains the predictions and analyses files for the five final models (because we used 5 different random seeds). The folder of the best performing model experiments/test_bert/cad_final_971242_BERT_0.01_0.00003_False
also contains the trained model.
Finetuning with different hyperparameters on train and dev
Used command:
python hatespeech_finetuning_multilabel.py tune BERT [output_dir]
Summary results are in: ../experiments/tune_bert_summary.txt
Best setting: learning rate 3e-5, weight decay 0.01
Finetune on train with selected hyper parameters, apply model on test
Run the models on train and test with the selected parameters
python hatespeech_finetuning_multilabel.py test BERT [output_dir]
Analysis of the results on the set set
Assuming that "../experiments/test_bert
is the output directory, read the log files and print macro/micro averages.
python analyze_transformers.py analyze_runs --input_pattern "../experiments/test_bert/cad_final_*_BERT_0.01_0.00003_False"
The results are provided in experiments/test_bert_summary.txt
Apply BERT again on the test set, now write predictions for each instance to an output file (test_output.txt
) in the model directory.
python analyze_transformers.py write_predictions --input_pattern "../experiments/test_bert/cad_final_*_BERT_0.01_0.00003_False" --model_type BERT
For each run, read in the predictions file and write out all the metrics:
python analyze_transformers.py print_metrics --input_pattern "../experiments/test_bert/cad_final_*_BERT_0.01_0.00003_False" --analysis_type "general_analysis"
This generates two files in each model folder: test_output_analysis.txt
and test_output_metrics.txt
.
Also run this once with analysis_type="label_analysis"
, this will output statistics related to the number of labels predicted and will generate test_output_label_analysis.txt
Now, by reading the file with evaluation metrics for each BERT model (we have 5, because we used 5 different random seeds), we can now write out mean and stds for each metric to standard output:
python analyze_transformers.py print_aggr_analysis --input_pattern "../experiments/test_bert/cad_final_*_BERT_0.01_0.00003_False" --metrics_file "test_output_metrics.txt"
The output is provided in ../experiments/test_bert/test_output_metrics_bert_aggregated.txt
Error analysis:
python analyze_transformers.py print_error_analysis --input_pattern "../experiments/test_bert/cad_final_*_BERT_0.01_0.00003_False"
@inproceedings{vidgen-etal-2021-introducing,
title = "Introducing {CAD}: the Contextual Abuse Dataset",
author = "Vidgen, Bertie and
Nguyen, Dong and
Margetts, Helen and
Rossini, Patricia and
Tromble, Rebekah",
booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
month = jun,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2021.naacl-main.182",
pages = "2289--2303",
abstract = "Online abuse can inflict harm on users and communities, making online spaces unsafe and toxic. Progress in automatically detecting and classifying abusive content is often held back by the lack of high quality and detailed datasets.We introduce a new dataset of primarily English Reddit entries which addresses several limitations of prior work. It (1) contains six conceptually distinct primary categories as well as secondary categories, (2) has labels annotated in the context of the conversation thread, (3) contains rationales and (4) uses an expert-driven group-adjudication process for high quality annotations. We report several baseline models to benchmark the work of future researchers. The annotated dataset, annotation guidelines, models and code are freely available.",
}
Data
contextual_abuse_dataset.py
: Loads in the dataset. Does a few preprocessing steps.check_data.py
: Contains methods to perform a few checks on the data.data_analysis.ipynb:
A notebook that explores the data and prints out statistics.
Notes:
- check_thread_structure: missing: 618, 02, 03
- Slurs are excluded from the experiments. If a post is only annotated as Slur, it becomes Neutral. Otherwise, the Slur label is just ignored.
- cad_27367 was excluded from the experiments because only the image is abusive.
- First time loading the dataset can give a
TypeError: argument of type 'Value' is not iterable
error.
Logistic Regression
lr.py
: Runs the logistic regression classifier (both tuning and testing). Writes the models and outputs to files.
Transformers
hatespeech_finetuning_multilabel.py
Script to finetune BERT and DistilBERT.distillbert_multilabel.py
adaptation of DistilBert to support multilabel classification.bert_multilabel.py
adaptation of Bert to support multilabel classification.analyze_transformers.py
analyze the results of the transformer models.
Utilities
classification_util.py
: Utilities for classification (writing output and evaluation metrics to file).classification_analysis.py
: Analyze classification results.
Tests
tests.py
: Checks a few processing methods.