zamling/PSALM

First stage scripts

Opened this issue · 2 comments

wzc9401 commented

Hi, does it include the first stage training script?

wzc9401 commented

or you're pretrained in the framework of llava code？

zamling commented

@wzc9401
Hi, this code do not contain the first training script. I follow the framework of llava to finish the first stage training with some small modifications:

  1. change ViT to swin-Transformer
  2. change mm_projector from nn.Linear to conv + linear