zhouxian/act3d-chained-diffuser

How to jointly optimize action detector and trajectory diffuser?

ManUtdMoon opened this issue · 6 comments

ManUtdMoon commented

Dear authors,

Thank you for your inspiring work!

I noticed that in the ChainedDiffuser paper your mentioned that "...train both the action detector and the trajectory diffuser jointly" and "we train the first 2 terms till convergence, and then add the 3rd term for joint optimization". However, I did not see that there are codes for joint optimization because the only model in main_trajectory.py is a DiffusionPlanner.

Would you please explain more about the actual joint training of Act3d and DiffusionPlanner?

Regards,
Dongjie

zhouxian commented

Hi, in our latest exps we found that simply training the models separately would yield similar/even better performance. Joint training is not needed any more.

ManUtdMoon commented

Hi, in our latest exps we found that simply training the models separately would yield similar/even better performance. Joint training is not needed any more.

Hi Zhou, thank you for your quick reply!

I think seperate training needs relabeling of the keypose. Could you please show me how it is done in the code？ Thank you！

zhouxian commented
ManUtdMoon commented

The paper mentioned that goal gripper pose is not ground truth but predicted by the action detector. Therefore, during training, I think the target keypose should be relabeled by act3d instead of taking from the ground truth. Is there something wrong with my understanding？

zhouxian commented
ManUtdMoon commented

Thank you for your answers！ Have a nice day！