Question about the model design.

[Yangr116]

Hi, this is a great work!
I would like to know why you use bidirectional Mamba? Does a single directional Mamba have any problems in your experiments?

[feizc]

Hi, It is generally believed that the single direction is not as good as the birectional Mamba. At the same time, different scan strategies can further improve the generationperformance, which can refer to the discussion in Zigma and DIM paper. For simplicity, we used bidirectional Mamba here.

However, it is worth noting that there has been an increasing focus of work on autoregression, such as llamagen [3] and Kaiming He' recent work [4].

[1] ZigMa: A DiT-style Zigzag Mamba Diffusion Model
[2] DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis
[3] Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
[4] Autoregressive Image Generation without Vector Quantization

[Yangr116]

thks! I have noticed these papers.😊 费政聪 ***@***.***>于2024年7月15日 周一11:16写道：

…

Hi, It is generally believed that the single direction is not as good as the birectional Mamba. At the same time, different scan strategies can further improve the generationperformance, which can refer to the discussion in Zigma and DIM paper. For simplicity, we used bidirectional Mamba here. However, it is worth noting that there has been an increasing focus of work on autoregression, such as llamagen [3] and Kaiming He' recent work [4]. [1] ZigMa: A DiT-style Zigzag Mamba Diffusion Model [2] DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis [3] Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation [4] Autoregressive Image Generation without Vector Quantization — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARTC2EH6DHHSTOVE35Y24SDZMM5J7AVCNFSM6AAAAABKK33J32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRXGYZTMMJQGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>