Enhancing Voice Deepfake Detection by Leveraging Hidden Layer Features from Pre-trained Model
Pytorch code for following paper:
- Title : Enhancing Voice Deepfake Detection by Leveraging Hidden Layer Features from Pre-trained Model
- Submited to Interspeech 2024 (write URL later)
- Autor : writed after Interspeech 2024
In the voice spoofing detection field, countermeasure (CM) sys- tems based on deep neural networks (DNNs), particularly those that leverage speech pre-trained models (PMs), have demon- strated promising performance. Despite these achievements, there is a lack of exploration into how to utilize the speech PM’s output and fine-tune it. This paper introduces hidden squeeze Conformer (HS-Conformer), a speech PM-based CM system that considers the above two points. The proposed sys- tem integrates the Wav2Vec 2.0 and the HM-Conformer with a newly proposed HS module. The HS module serves as a critical bridge, squeezing task-appropriate information and capturing spoofing evidence based on an attention mechanism from the PM’s multiple hidden outputs. When evaluated on the ASVspoof 2021 deepfake evaluation partition, the HS- Conformer outperforms recent CM systems by 26%, achieving a state-of-the-art equal error rate of 1.92%.
We used 'nvcr.io/nvidia/pytorch:23.08-py3' image of Nvidia GPU Cloud for conducting our experiments.
- Python 3.8.12
- Pytorch 2.1.0a0+29c30b1
- Torchaudio 2.0.1
- transformers[torch]
- huggingface_hub
- torch-audiomentations
- Make dockerfile image
# build docker img
# run at /~/HS-Conformer
./docker/build.sh
- Run docker image
sudo nvidia-docker run -it --rm --ipc=host --shm-size 20G \
-v {path_DB}:/data \
-v {path_HS-Conformer}/env202308:/environment \
-v {path_HS-Conformer}/env202308/results:/results \
-v {path_HS-Conformer}:/code env202308:latest
# CAUTION! You need to write your path
# PATH_DB
# |- ASVspoof2019
# | |- LA
# |- ASVspoof2021_DF
# |- ASVspoof2021_DF_eval
# |- keys
First, you need to set system arguments. You can set arguments in arguments.py
. Here is list of system arguments to set.
- Caution: This code only run on multi-GPUs!
1. 'usable_gpu' : {Available_GPUs}
'usable_gpu' is the order of the GPUs you have available.
input type is str # ex) '0,1'
CAUTION! You need to use 2 or more GPUs
We have a basic logger that stores information in local. However, if you would like to use an additional online logger (wandb or neptune):
- In
arguments.py
# Wandb: Add 'wandb_user' and 'wandb_token'
# Neptune: Add 'neptune_user' and 'neptune_token'
# input this arguments in "system_args" dictionary:
# for example
'wandb_group' : 'group',
'wandb_entity' : 'user-name',
'wandb_api_key' : 'WANDB_TOKEN',
'neptune_user' : 'user-name',
'neptune_token' : 'NEPTUNE_TOKEN'
- In
main.py
# Just remove "#" in logger
# logger
builder = egg_exp.log.LoggerList.Builder(args['name'], args['project'], args['tags'],
args['description'], args['path_scripts'], args)
builder.use_local_logger(args['path_log'])
# builder.use_neptune_logger(args['neptune_user'], args['neptune_token'])
# builder.use_wandb_logger(args['wandb_entity'], args['wandb_api_key'],
# args['wandb_group'])
logger = builder.build()
logger.log_arguments(experiment_args)
python /code/HS-Conformer/system_code/main.py
Please cite this paper if you make use of the code.
# add later...