Allow passing in a pre-existing TokenEmbedding into TransformerWrapper
Waino opened this issue · 0 comments
Waino commented
Allow passing in a pre-existing TokenEmbedding into TransformerWrapper
Currently TransformerWrapper always creates a new TokenEmbedding object.
TransformerWrapper should take an optional keyword argument token_emb
.
If not specified or set to the default value None
, a new TokenEmbedding
is created exactly as before. If a pre-existing TokenEmbedding
is specified, it is used as the token_emb
.
This feature is useful for parameter sharing, allowing the use of a single shared embedding with multiple Transformers. I have a use case for this feature in modular multilingual machine translation.