Question: Shouldn't learnable positional embeddings be MuReadoutLayers ?

[codedecde]

Hi !
I was wondering if (learned) positional embeddings should be MuReadout layers (

mutransformers/mutransformers/models/bert/modeling_bert.py

Line 174 in 480287c

self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)

), since they map to a finite dimensional space ?

Would be grateful for any advice :)

Thank you !