LiNeS fine-tuning method for XTTS v2 gpt finetuning
kunibald413 opened this issue · 0 comments
kunibald413 commented
quote from paper
post-training technique that linearly rescales the
updates of different layers in the task vector based on their depth in the network. LiNeS is designed
to retain general features in the shallow layers while preserving the task-specific adaptations in the
deeper layers
could you help me understand if the method mentioned is applicable for the xtts v2 model?
Would you think it's straight forward to implement and/or give some hints for that? I don't have that much experience in ML.
Additional context
code:
wang-kee/LiNeS
paper:
arxiv.org/abs/2410.17146