Support for the `Embedding` module
ShayekhBinIslam opened this issue · 1 comments
ShayekhBinIslam commented
DoRA is not currently supported when new tokens are added and embedding layer needs to be retrained. Is there any solution?
nbasyl commented
Hi, for retraining the embedding layer, I recommend using standard LoRA, as DoRA has not yet been tested on retraining the embedding layer.