Clarification of the settings on "train split", to reproduce validation result

[ivanpp]

Hello, thank you for your fantastic work and released code. I have some questions regarding the setting used to reproduce the result on validation set.

Max epoch

In your paper (https://arxiv.org/pdf/2312.14474), chapter 4.1, inside implementation details, you suggested:

• Without the MixUp3D, we train the model for 150 epochs, after which overfitting occurs.
• While using MixUp3D, the model can be trained for 600 epochs without overfitting.

Here 150 or 600 epochs seems to be an issue of whether using "MixUp3D" or not.

However, in one of your issue (#2 (comment)), you mentioned:

• In order to compare fairly with other methods, we use 150 epochs for training on the train set.

I would like to to reproduce the result in your paper that trains on training set and evaluates on validation set:

• what max_epoch did you use?
• what decay_list did you use?
• If there are other different hyperparameter setting?

When to start LSS

In your paper (https://arxiv.org/pdf/2312.14474), chapter 4.1, inside implementation details, you suggested:

• The LSS starts after 0.3 × total epochs (experimental parameters) for warmup

However, in your code you set it to 100:

MonoLSS/lib/losses/loss_function.py

Lines 128 to 137 in 3d1351e

	if self.epoch < 100:
	vis_depth_loss = torch.mean(vis_depth_loss)
	else:
	attention_map_mask = attention_map.view(-1,49)
	if self.epoch<0:
	attention_map_mask = gumbel_softmax_topk(attention_map_mask, tau=1, hard=True)
	else:
	attention_map_mask = gumbel_softmax_topk(attention_map_mask,tau=1,hard=True,soft_ = True)
	attention_map_mask = attention_map_mask.view(-1,7,7)
	vis_depth_loss = torch.mean(vis_depth_loss*attention_map_mask) * 10

Here I would like to ask:

• To reproduce your validation set result, what number should I set here?

Thank you in advance for your clarification and guidance,
ivanpp

[jjrCN]

Trains on training set and evaluates on validation set:

1. max_epoch = 150
2. decay_list = [90, 120]
3. warm up epoch = 50, that means change the code in loss_function.py(line 128): "if self.epoch < 50: "
4. change split = 'trainval' to 'train' in dataloader_helper.py

Thank you for your attention, remember to pull the latest code.

[ivanpp]

Trains on training set and evaluates on validation set:

1. max_epoch = 150
2. decay_list = [90, 120]
3. warm up epoch = 50, that means change the code in loss_function.py(line 128): "if self.epoch < 50: "
4. change split = 'trainval' to 'train' in dataloader_helper.py

Thank you for your attention, remember to pull the latest code.

Thank you so much for your quick reply. Following this setting, I have been able to reproduce your validation results.
One more question regarding the test set setting.
I assume that the code you uploaded initially is used for test set, with the following hyperparameters:

1. max_epoch: 600
2. decay_list: [240, 360, 480]
3. lss starts epoch: 100

Is it correct and complete by just run your default code with default yaml file? Or I should follow the paper to set "lss start epoch" to 600*0.3=180? Thank you again for your time.

[jjrCN]

Yes, default code is for test set. You can run trainning directly and do not need to change anything. Using the default yaml file is correct and complete.