question about fashionbert

[tjulyz]

Thanks for sharing your code! I have the following questions about fasionbert.

(1) Have you evaluated the performance of fasionbert without pretraining? That is, training a model by removing mlm and mpm only with image-text-matching task. Besides, There are not fine-tuning on downstream task (eg. image-text matching). So why do you call that as pertaining instead of training.

(2) Have you resize the image patch (3232->224224) for image feature extraction?

(3) Which backbone network is selected for image feature extraction in your released pertained model? resnet50 or resnext101?

[search-opensource-space]

(1) We do not attempt to train fashionbert only with image-text-matching task. Since we evaluate fashionbert model with fashion-gen dataset, we donot perform fine-tuning again.

(2) Need to resize the image patch to fit the feature extraction.

(3) Resnet50

[tjulyz]

Thanks a lot!
Do you use the specific masks for image patche in training set? Besides, in your code, a pertained imagebert is used for initializing the fashionbert. Which dataset the imagebert is pretrained based on?

[dannygao1984]

Thanks a lot!
Do you use the specific masks for image patche in training set? Besides, in your code, a pertained imagebert is used for initializing the fashionbert. Which dataset the imagebert is pretrained based on?

1. randomly mask the image patches
2. fashionbert is continuely pretrained based on google bert(base) with fashionGen dataset

[tjulyz]

I am confused about the pertained model you loaded in the code

where the 'pretrain_model_name_or_path=pai-imagebert-base-en' instead of ‘= google-bert-base-en’

[jerryli1981]

When you run_train.sh first time, ez will automatically download model to modelZooBasePath. Pleas check your os.getenv("HOME") / eztransfer_modelzoo.
flags.DEFINE_string("modelZooBasePath", default=os.path.join(os.getenv("HOME"), ".eztransfer_modelzoo"), help="eztransfer_modelzoo")

[tjulyz]

Yes. What I am confused is about the downloaded model.

In the readme file, the command for pretrained model is 'pretrain_model_name_or_path=pai-imagebert-base-en' instead of ‘= google-bert-base-en’. Should I change it to googlebert when making pretrain on FashionGen?

[jerryli1981]

You can conduct two experiments to compare the two pretrained models.

[HowieMa]

Thanks a lot!
Do you use the specific masks for image patche in training set? Besides, in your code, a pertained imagebert is used for initializing the fashionbert. Which dataset the imagebert is pretrained based on?

1. randomly mask the image patches
2. fashionbert is continuely pretrained based on google bert(base) with fashionGen dataset

Thank you for your excellent work. May I ask one following-up question about FashionBERT?
Could you please give some details about the ratios of positive pairs and negative pairs during the training process? Like in one mini-batch, 50% of examples are positive and the rest are negative. Thanks a lot!