code for "U-shape" attention difference.
Weifeng-Chen opened this issue · 4 comments
how do you calculate the difference of timesteps to get the "U-shape". I try to re-implement it and found that from Timestep 1000 to 0, the difference become larger... Can you provide an example to get it or more detail about how to get the "U-shape" curve?
we compare the output of attention with previous step of same layer. it works for all models we test on
we compare the output of attention with previous step of same layer. it works for all models we test on
got it, thanks. I re-implement in opensora and it indeed "u-shape". we don't get similar results in our inhouse model. But thanks for the insight and we'll do more test.
我们将注意力的输出与同一层的上一步进行比较。它适用于我们测试的所有模型
we compare the output of attention with previous step of same layer. it works for all models we test on
Hello, I would like to ask, so when you compare the differences in the two diffusion steps are you averaging the results of the differences in each layer to represent them?
not for me