Multidecoder Discussion

[dongfangyixi]

Hi, I have tried Single Decoder and Multi decoder with EM, random select decoder, and select all decoder.
But Single decoder is always better than multi decoder.
Did any one have the save issue with me?
I was wondering how to set the multi decoder config to have a better result than single decoder.

[stepankonev]

Hi! Could you share your configs? Don't really get what are these: random select decoder, and select all decoder. How do you compare single and multi decoder?

[dongfangyixi]

Hi,

1. "select all decoder" means that I set line 213 in modules:
   random_head_selector = np.ones_like(random_head_selector) * (random_head_selector > 0.5)
   to
   random_head_seletor = np.ones_like(random_head_selector)

2. The config file of Single decoder is "final_RoP_Cov_Single.yaml", Multidecoder is "final_RoP_Cov_A_fMCG.yaml"

3. I use waymo_open_dataset.metrics to calculate the map minade etc. metrics from the predicted trajectories of both model.
   Besides, I submit the validation set results to the waymo motion challenge webset to get the metrics for second check.
   The result of multidecoder is less than single decoder for 3 or 4 points in softmap.

[stepankonev]

I see. Did EM work well for you? I had some issues with the numerical stability while using it. You also probably should train models with multiple decoders for longer