openai/sparse_attention

Questions about novelty

zhaoguangxiang opened this issue · 2 comments

zhaoguangxiang commented

The paper is well written and makes great results in various datasets.
However, the contribution of novelty is unclear.
Q1： How is the Sparse Transformer (strided) different from local attention?
Q2： How is the Sparse Transformer (fixed) different from block self-attention? ( ICLR 2018 https://openreview.net/forum?id=H1cWzoxA-)?

alphadl commented

seems that the author is busy ~