KennethEnevoldsen/snp-compression

Grid search on activation function

Opened this issue · 0 comments

KennethEnevoldsen commented

Seems like most people today don't use reLU. I should probably test out e.g. GELU (used in Wav2Vec2.0) or other alternatives.