bigscience-workshop/multilingual-modeling

Issues

• Implement “Composable Sparse Fine-tuning for Cross-Lingual Transfer”

  #23 opened 2 years ago by haileyschoelkopf
  0

• Implement (IA)^3 in T-Few

  #39 opened 2 years ago by yongzx
  1

• Implement "Adaptable Adapters"

  #34 opened 2 years ago by haileyschoelkopf
  2

• Calculate Trainable Parameters Correctly for `extend` vocab strategy.

  #42 opened 2 years ago by yongzx
  0

• BitFit Finetuning.

  #21 opened 2 years ago by yongzx
  2

• [Bug, needs reproduction] [low priority] Weird loss behavior with `pfeiffer+inv` config

  #40 opened 2 years ago by haileyschoelkopf
  1

• Package the code up neatly to pass to JZ.

  #29 opened 2 years ago by yongzx
  1

• Implement Ladder Side-Tuning

  #35 opened 2 years ago by haileyschoelkopf
  1

• [Low Prio] Implement Diff-Pruning

  #37 opened 2 years ago by yongzx
  0

• [Lower Priority] Implement "Fisher Induced Sparse uncHanging (FISH) Mask"

  #38 opened 2 years ago by haileyschoelkopf
  0

• Make generation task (XLSUM) work for decoder-only model and follow the original setting.

  #26 opened 2 years ago by yongzx
  0

• Trainable Parameters in madx_clm_run.py are incorrect for `extend` strategy.

  #36 opened 2 years ago by yongzx
  0

• Implement LoRA.

  #31 opened 2 years ago by yongzx
  1

• Control Extra Params (use Adapter 16x reduction size as control)

  #25 opened 2 years ago by yongzx
  0

• Implement Prefix-Tuning.

  #30 opened 2 years ago by yongzx
  0

• Unable to train Burmese tokenizer for training_size = 100,000

  #18 opened 2 years ago by yongzx
  5

• Add support for only the final task-specific layers in eval script

  #24 opened 2 years ago by yongzx
  0

• Inconsistent Evaluation Results

  #17 opened 2 years ago by yongzx
  0

• Adding Language Post-Training: Evaluation Suite

  #5 opened 2 years ago by yongzx
  1

• Adding Language Post-Training: MAD-X Adapters

  #6 opened 2 years ago by yongzx
  1

• Adapter Training - Frozen transformer.wpe.weight?

  #9 opened 2 years ago by yongzx
  1

• Incrementally adding new languages to multilingual checkpoint

  #7 opened 2 years ago by vnikouliNLE
  2

• Incrementally adding new languages to pre-trained models

  #2 opened 2 years ago by hadyelsahar
  9

• Adding Language specific validation sets to deepspeed

  #1 opened 3 years ago by hadyelsahar
  4