shamanez/Self-Supervised-Embedding-Fusion-Transformer

Replication issues

Dayire opened this issue · 3 comments

Dayire commented

Dear Authors,

Thank you for sharing your software with us. I am trying to replicate your results but I am having the following issues/comments:

  1. Am I correct to use those pre-processed files from your other Repo (BERT-like-is-All-You-Need)? link . If not, can you share the pre-processed data or how to preprocess it from the raw one?
  2. If the first step is correct, I then used mosi_data as the path to the raw dataset by modifying raw_audio_text_video_dataset.py to load directly the data path root (I did not find any command arguments to input that). So far, this works for mosi
  3. training runs but sadly, the loss does not decrease, I'm using a batch size of 16 for mosi and I have 64GB of GPU memory (4*16)

Is there something I am missing? I'd appreciate your help.

shamanez commented
Dayire commented

Thank you, this did help.

Another question, according to the paper, I need to change the following hyper parameters (among others) between MOSI/CMU_MOSI (currently interested in):
Self Attention Blocks, IMA Blocks, Self Attention Heads, IMA Heads

Can you tell me which one are those from the Training Command Line? Or guide me here? I looked everywhere in the code but couldn't find them.

Appreciate!

shamanez commented

Please check the README. I have added the command-line training.
https://github.com/shamanez/Self-Supervised-Embedding-Fusion-Transformer#training-command

Mos MOSI and MOSEI, use --regression-target-mos