FTT: Forecasting Temporal Trends for Fake News Detection

Official repository for Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detection, which has been accepted by ACL 2023.

Introduction

The proposed FTT is a model-agnostic framework, which could tackle the temporal generalization issue by adjusting the training data distribution to be closer to future data distribution.

Dataset

Due to commercial restrictions, we cannot release the complete dataset. We have placed 500 samples at roll_seasonal_data/roll_online_bf20_demo so you can complete the whole training process. You can create your own dataset following the sample format.

Code

File Tree

.
├── grid_search.py
├── logs  # results save here
│   └── seaonal_res_analyse.ipynb
├── main.py
├── models
│   ├── bert.py
│   ├── eann.py
│   └── layers.py
├── requirements.txt
├── reweight_roll_season
│   ├── get_embeddings.py
│   ├── predict_freq.py
│   ├── roll_seasonal_data  # sample data here
│   ├── roll_seasonal_user_data  # adjusted data save here
│   ├── run.sh  # Step 1 running script
│   ├── single_pass_cluster.py
│   ├── utils.py
│   └── weight_score_cal.py
├── run.sh  # Step 2 running script
└── utils
    ├── dataloader.py
    └── utils.py

Requirements

Refer to requirements.txt

You can run pip install -r requirements.txt to deploy the environment quickly.

Pretrained Models

You can download pretrained models (sentence-roberta-wwm-ext and chinese-bert-wwm-ext) and change paths (sentence_transformer_path and bert_path) in the corresponding scripts.

Run

The experiment consists of two steps. In Step 1, adjusted training data is generated using the FTT method. In Step 2, the detection model is trained based on the adjusted training data.

Step 1: Generate Adjusted Training Data Using FTT

FTT consists of four steps, which are News Representation, Topic Discovery, Temporal Distribution Modeling and Forecasting and Forecast-Based Adaptation. You can complete these four steps by adjusting the parameters in the reweight_roll_season/run.sh script and executing the script.

Parameter Configuration:

data_name: data name under reweight_roll_season/roll_seasonal_data
gpu: the index of gpu you will use
cluster_threshold: the similarity threshold (denoted as $θ_{sim}$ in the paper) to determine when to add a new cluster
predict_threshold: the count threshold (denoted as $θ_{count}$ in the paper), we do not consider the clusters with news items less than it
reweight_threshold: the mape threshold (denoted as $θ_{mape}$ in the paper), we remove the topics which have a mean absolute percentage error (MAPE) larger than it
thres_low & thres_high: the reweight-bound (denoted as $θ_{lower}$ and $θ_{upper}$ in the paper), we set the weight smaller than thres_low and larger than thres_high as thres_low and thres_high, respectively, to avoid the instability during the training process

Step 2: Training the Detection Model Based on the Adjusted Training Data

You can complete the training process by adjusting the parameters in the run.sh script and executing the script.