Benefits of using S4D?

[aga-relation]

Hi! Very nice paper, congratulations :) I was just wandering - I saw you had S4D implemented in your repo but I couldn't find any related results in the paper. Do you have any insight whether this model has benefits for modelling DNA or not necessarily?

Thank you very much!

[gonzalobenegas]

Hello, thank you for your interest in our paper!

We did a very quick try of S4D but without really evaluating different hyperparameter choices, which seem to be important. It was doing a bit worse than our ConvNet so we didn't pursue further.

My guess would be that choosing the right training dataset will have a much bigger impact than the specific model architecture. Also, 512 context size might be too small for S4D to shine.