Question about weight value of key and query in SA Layer

Question

Question about weight value of key and query in SA Layer

dogyoonlee opened this issue 4 years ago · 1 comments

dogyoonlee commented 4 years ago

Hello,

I really appreciate to your creative work.

However, I hope to know why did you use same weight value when initialize the q_conv and k_conv kernel?

As I know, they don't have to be identical.

Is there any reason?

Thank you for your work again.

Answer 1 · 2021-02-04T15:09:13.000Z

Hi,
thank you for your attention to PCT.
In experiment, we find it can make the entire network converge better. Moreover, we observe that the weights of q_conv kernel and k_conv kernel are different after training.