BlinkDL/RWKV-LM

Probable mistake in Eq. 16 in the preprint

Opened this issue · 2 comments

Thank you for the great work.

I have checked the code and the paper, and I believe that the "lerp" in Eq. 16 might actually be "ddlerp".

I also find the code uses ddlerp method. I also think Eq.19 also doesn't match the Table 7.

@zeyun-zhong You are correct, thanks for noticing this! We will update the paper.

@UEA-GCG Thank you for finding this error. I believe you are pointing out that the cumprod in Eq. 19 is indexed incorrectly. (It should go from j=i+1 to t-1, since it's really a reverse cumprod). Please let me know whether you see some other issue(s) as well with this formula.