LASSL 모델 실험
tucan9389 opened this issue · 0 comments
tucan9389 commented
RoBerta
| model | classifier-lr | upstream-lr | epoch | best f1-score |
|---|---|---|---|---|
| lassl/roberta-base | 1e-3 | 1e-3*0.01 | 300 | 0.756 |
| lassl/roberta-base | 5e-3 | 5e-3*0.01 | 300 | 0.822 |
| lassl/roberta-base | 1e-2 | 1e-2*0.01 | 300 | training... |
GPT2
| model | classifier-lr | upstream-lr | epoch | dropout | bestf1-score |
|---|---|---|---|---|---|
| lassl/gpt2-base | 1e-3 | 1e-3*0.01 | 300 | 0.5 | 0.706 |
| lassl/gpt2-base | 1e-3 | 1e-3*0.01 | 300 | 0.1 | 0.69 |
| lassl/gpt2-base | 1e-2 | 1e-2*0.01 | 300 | 0.5 | 0.670 |
| lassl/gpt2-base | 1e-2 | 1e-2*0.01 | 300 | 0.1 | 0.668 |
| lassl/gpt2-base | 5e-3 | 5e-3*0.01 | 300 | 0.1 | 0.688 |