amaralibey/Bag-of-Queries

The Hyper-parameters for training the DinoV2+BoQ?

Closed this issue · 2 comments

Frost24K commented

Hi,
Great work! Can you provide the concrete information in training BoQ with DinoV2 backbone? Such as the learning rate, scheduler options, train epochs, etc. Thank you!

amaralibey commented

Hi @Frost24K,

The model we shared has been trained on GSV-Cities with:

  • backbone: dinov2_vitb14 (unfreeze last 2 blocks)
  • batch size: 160x4
  • image size: 280x280
  • lr: 0.0002
  • optimizer: AdamW
  • warmup: 10 epochs (linear warmup)
  • lr_schedul: x0.1 every 10 epochs
  • total epochs: 40 (converges way before, between 20-30).
  • data augmentation: RandAugment(num_ops=3)

The VRAM usage when employing batches of size 160x4 is ~31GB. However, I've got similar performance using batch size of 100x4, which requieres ~19GB of VRAM.

Best,
Amar

  • 👍1
Frost24K commented

Hi @amaralibey,
Thank you for your prompt response !

  • ❤️1