airaria/TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
PythonApache-2.0
Issues
- 2
学生模型权重初始化问题
#121 opened by cgh-code777 - 3
请问支持BERT-of-Theseus的蒸馏方式吗
#120 opened by zhanghanweii - 2
麻烦问下,目前支持llama模型吗
#119 opened by StevensPrime - 2
可以使用chatgpt蒸馏到bert或者T5吗?
#118 opened by Hoogck - 5
- 12
notebook_examples/msra_ner.ipynb 运行报错
#112 opened by MrRace - 2
老师您好,我想问一下,比如roberta蒸馏到tinybert,中间的hidden是通过线性层拉到同样的维度去算mse,那在推理的时候岂不是这些经过梯度更新的线性层毫无作用?那请问这些线性层仅仅就是为了调整维度?
#116 opened by lean-wang - 4
老师,您好,请问有多任务多教师的蒸馏的demo吗?
#115 opened by lean-wang - 2
msra_ner.ipynb最后的trainer.evaluate()显示CUDA out of memory,请问训练的显存要求是多大?十分感谢!
#114 opened by jinxiaolinlin - 7
msra_ner.ipynb复现代码bug
#85 opened by HXYstudy - 4
不同维度蒸馏有对应的例子吗,从768降到256
#113 opened by weidalan - 2
关于ner数据的处理
#111 opened by Soulscb - 7
在VisionTransformer
#110 opened by zym1599 - 2
Does it support translation model?
#108 opened by AIikai - 2
How about the distillation effect of gpt2 ?
#107 opened by xk503775229 - 3
Picking right layers
#106 opened by patryk-at-pieces - 5
interpreting intermediate matches
#103 opened by kaliaanup - 3
Show the progress bar when training.
#104 opened by Gridnn - 2
- 3
pre-trained student weights
#101 opened by roymiles - 4
TextBrewer/src/textbrewer/distiller_utils.py get_outputs_from_batch fails tocheck dicts properly for maskedLM
#98 opened by AddedK - 1
- 1
请问可以直接用于unilm中的NLU和NLG吗?
#96 opened by cingtiye - 0
如何实现early stopping
#95 opened by yuange555 - 3
请问有添加早停机制的打算吗?
#94 opened by catqaq - 2
如何蒸馏不分层的新特征?
#93 opened by catqaq - 3
请问有针对BertForMaskedLM的蒸馏示例吗
#92 opened by dongteng - 5
- 7
mnli main_train
#88 opened by Soulscb - 1
关于任务无关的蒸馏
#89 opened by savannahfan - 1
- 6
中间层Loss,会去更新后面网络的参数吗
#86 opened by DvHuang - 3
是否能做预训练的蒸馏
#87 opened by YoungErm - 2
- 4
- 5
random_token_example error
#81 opened by tanyaroosta - 1
Notebook JSON is invalid
#82 opened by tanyaroosta - 2
PyTorch Lightning
#80 opened by tchaton - 2
examples/random_token_example, when I run python distill.py then exception Killed
#79 opened by dulante00 - 2
CUDA Error with your Notebook Example
#78 opened by cabisarri - 0
Cuda Error in scripts
#77 opened by cabisarri - 2
使用自定义的网络结构
#76 opened by zhangatao - 2
关于MNLI任务复现问题
#72 opened by sunnan-nn - 2
请问此框架的loss函数是否存在问题?
#71 opened by Jay2Coomzz - 4
模型没有被训练,每个epoch保存的模型weight一模一样。
#70 opened by Jay2Coomzz - 6
GeneralDistiller的train函数报错
#69 opened by Jay2Coomzz - 5
关于中文阅读理解数据集t4学生模型蒸馏配置的问题
#68 opened by SouthBays - 15
examples/mnli_example: run_mnli_train.sh 模型没被训练
#67 opened by Yin169 - 4
Data preparation
#66 opened by liuhl-source - 4