Reproducibility

[AliLotfi92]

1. I get the same results no matter --optimization is 0 or 1.
   --optimization=1 (is your method)
   --optimization=0
   python neural_sim_main.py --config ../configs/nerf_param_ycbv_general.txt --object_id 2 --expname exp_ycb_synthetic --psi_pose_cats_mode 5 --test_distribution 'one_1' --optimization 1

python neural_sim_main.py --config ../configs/nerf_param_ycbv_general.txt --object_id 2 --expname exp_ycb_synthetic --psi_pose_cats_mode 5 --test_distribution 'one_1' --optimization 0

Correct me if I'm wrong!

2. It seems the object detector initialized based on the optimal weights of the previous iteration. Is that right? If yes, what is the logic behind that? Could the results in the paper are based on the over training of object detector?

3. Will you release the code for

multiple objects, I mean some code to verify the results of Table 1.
the code for illumination or zoom (as it has been asked by other users, too)

Thanks again for such a wonderful work. It would be a much better contribution in ML if you release a code that the research community can use more effectively.

[gyhandy]

Thanks for your questions.

(1) Based on the command line, the interest object is id-2 (cheez-it box), so object id-2 has train pose bin-5 and test bin-1, while all other five objects have no train/val post domain gap, so the optimization does not influence their accuracy. For object id-2, the accuracy is already very high (near 100%), so the optimization may not be helpful.

I recommend you try object id-11 (a pitcher). Please also try different initial poses and test poses. Sometimes if the initial pose is similar to the test pose, then adding optimization may not help much. Also, training parameters may influence the results.

(2) Initialized based on previous iterations or not is orthogonal to our optimization. Training from scratch in each epoch will take a longer time. Initialization from previous epochs could still show the effectiveness of optimization because the baseline will use the same setting except for adjusting the pose parameter by optimization.

(3) Yes, we plan to release these codes. Sorry for the lateness.

We added the license. Thanks for your suggestion!

Thank you again for your interest in our work.

[AliLotfi92]

Thanks for getting back!

The below figure is the results for --object_id 11 based on these two commands;

- python neural_sim_main.py --config ../configs/nerf_param_ycbv_general.txt --object_id 11 --expname exp_ycb_synthetic --psi_pose_cats_mode 5 --test_distribution 'one_1' --optimization 1
- python neural_sim_main.py --config ../configs/nerf_param_ycbv_general.txt --object_id 11 --expname exp_ycb_synthetic --psi_pose_cats_mode 5 --test_distribution 'one_1' --optimization 0

Still, I'm not able to reproducible results. Can you please clarify?

Thanks for your consideration.

[AliLotfi92]

Is there any update, or suggestions?

[ratheile]

@AliLotfi92 May I ask on an update? Could you finally reproduce the results?

[AliLotfi92]

@ratheile, No! Do you have the same issue with the released code?

[ratheile]

Currently testing, I will report back later!

[AliLotfi92]

okay, let me know if you see any difference between conventional training and this method, thanks!

[AliLotfi92]

@ratheile any update?