facebookresearch/swav

Is this a reasonable graph for the losses?

sarmientoj24 opened this issue · 0 comments

sarmientoj24 commented

I am using the same thing for 8 GPU

 base_lr: 0.6
                                     batch_size: 32
                                     checkpoint_freq: 25
                                     crops_for_assign: [0, 1]
                                     data_path: /home/XXX/swav/swav/data
                                     dist_url: env://
                                     dump_checkpoints: dump/test2_ep400_b32_sz_320_128/checkpoints
                                     dump_path: dump/test2_ep400_b32_sz_320_128
                                     epoch_queue_starts: 15
                                     epochs: 400
                                     epsilon: 0.05
                                     feat_dim: 128
                                     final_lr: 0.0006
                                     freeze_prototypes_niters: 5005
                                     gpu_to_work_on: 0
                                     hidden_mlp: 2048
                                     is_slurm_job: False
                                     local_rank: 0
                                     max_scale_crops: [1.0, 0.2]
                                     min_scale_crops: [0.2, 0.1]
                                     nmb_crops: [2, 6]
                                     nmb_prototypes: 3000
                                     queue_length: 3840
                                     rank: 0
                                     seed: 31
                                     sinkhorn_iterations: 3
                                     size_crops: [280, 120]
                                     start_warmup: 0
                                     sync_bn: pytorch
                                     syncbn_process_group_size: 8
                                     temperature: 0.1
                                     use_fp16: True
                                     warmup_epochs: 0
                                     wd: 1e-06
                                     workers: 10
                                     world_size: 8

Losses

image