HS-Conformer

Enhancing Voice Deepfake Detection by Leveraging Hidden Layer Features from Pre-trained Model

Info

Pytorch code for following paper:

Title : Enhancing Voice Deepfake Detection by Leveraging Hidden Layer Features from Pre-trained Model
- Submited to Interspeech 2024 (write URL later)
Autor : writed after Interspeech 2024

Abstract

In the voice spoofing detection field, countermeasure (CM) sys- tems based on deep neural networks (DNNs), particularly those that leverage speech pre-trained models (PMs), have demon- strated promising performance. Despite these achievements, there is a lack of exploration into how to utilize the speech PM’s output and fine-tune it. This paper introduces hidden squeeze Conformer (HS-Conformer), a speech PM-based CM system that considers the above two points. The proposed sys- tem integrates the Wav2Vec 2.0 and the HM-Conformer with a newly proposed HS module. The HS module serves as a critical bridge, squeezing task-appropriate information and capturing spoofing evidence based on an attention mechanism from the PM’s multiple hidden outputs. When evaluated on the ASVspoof 2021 deepfake evaluation partition, the HS- Conformer outperforms recent CM systems by 26%, achieving a state-of-the-art equal error rate of 1.92%.

Prerequisites

Environment setting

We used 'nvcr.io/nvidia/pytorch:23.08-py3' image of Nvidia GPU Cloud for conducting our experiments.

Python 3.8.12
Pytorch 2.1.0a0+29c30b1
Torchaudio 2.0.1
transformers[torch]
huggingface_hub
torch-audiomentations

Make dockerfile image

# build docker img
# run at /~/HS-Conformer
./docker/build.sh

Run docker image

sudo nvidia-docker run -it --rm --ipc=host --shm-size 20G \
-v {path_DB}:/data \
-v {path_HS-Conformer}/env202308:/environment \
-v {path_HS-Conformer}/env202308/results:/results \
-v {path_HS-Conformer}:/code env202308:latest

#	CAUTION! You need to write your path
# PATH_DB
#   |- ASVspoof2019
#   |      |- LA
#   |- ASVspoof2021_DF
#          |- ASVspoof2021_DF_eval
#          |- keys

Run experiment

Set system arguments

First, you need to set system arguments. You can set arguments in arguments.py. Here is list of system arguments to set.

Caution: This code only run on multi-GPUs!

1. 'usable_gpu'	 : {Available_GPUs}
	'usable_gpu' is the order of the GPUs you have available.
	input type is str # ex) '0,1'

	CAUTION! You need to use 2 or more GPUs

Additional logger

We have a basic logger that stores information in local. However, if you would like to use an additional online logger (wandb or neptune):

In arguments.py

# Wandb: Add 'wandb_user' and 'wandb_token'
# Neptune: Add 'neptune_user' and 'neptune_token'
# input this arguments in "system_args" dictionary:
# for example
'wandb_group'   : 'group',
'wandb_entity'  : 'user-name',
'wandb_api_key' : 'WANDB_TOKEN',

'neptune_user'  : 'user-name',
'neptune_token' : 'NEPTUNE_TOKEN'

In main.py

# Just remove "#" in logger

# logger
builder = egg_exp.log.LoggerList.Builder(args['name'], args['project'], args['tags'], 	
                                         args['description'], args['path_scripts'], args)
builder.use_local_logger(args['path_log'])
# builder.use_neptune_logger(args['neptune_user'], args['neptune_token'])
# builder.use_wandb_logger(args['wandb_entity'], args['wandb_api_key'], 
# 												 args['wandb_group'])
logger = builder.build()
logger.log_arguments(experiment_args)

Now you can run our code

python /code/HS-Conformer/system_code/main.py

Citation

Please cite this paper if you make use of the code.

# add later...

talkingnow/HS-Conformer