openai/finetune-transformer-lm
Code and model for the paper "Improving Language Understanding by Generative Pre-Training"
PythonMIT
Issues
- 1
What is training accuracy of the language model?
#32 opened by peace195 - 0
The Conv1d over Linear?
#49 opened by maciejbalawejder - 10
Error!! :(
#39 opened by amirakazadeh - 0
lm_h = tf.reshape(h[:, :-1], [-1, args.n_embd])
#48 opened by lshowway - 0
what does the variable n_ctx mean ?
#47 opened by lshowway - 0
great dataset
#46 opened by amirakazadeh - 0
- 3
how could i train the language with chinese
#20 opened by wykdg - 0
- 0
Why do we need to apply mask while fine tuning?
#43 opened by pranoy-k - 0
Is it possible to edit the code based on this project to train from scratch?
#42 opened by guotong1988 - 1
How to pretrain the model?
#27 opened by ShuGao0810 - 4
Where can I get these two datasets?
#41 opened - 1
have you ever try a bigger corpus
#38 opened by ZizhenWang - 0
Using it as a Language model
#37 opened by mdasadul - 0
About the non-determinism due to GPU ops
#36 opened by OanaMariaCamburu - 0
- 1
help to understand bpe logic
#24 opened by BogdanDidenko - 0
- 0
What is the specific formula for learning rate used in adam optimizer during pre-train?
#29 opened by ShuGao0810 - 0
- 6
CPU support
#3 opened by throwaway73 - 1
Source Exhausted
#26 opened by Brandonnogithub - 1
- 0
Universal Transformer as base architecture
#23 opened by rodgzilla - 4
Cannot reproduce this experiment
#15 opened by Polynomia - 2
BUG in utils.py
#21 opened by YichenGong - 1
Convolution layer filter width
#16 opened by sharpsy - 2
Using conv1d with kernel size 1
#19 opened by ollmer - 1
- 9
Cannot reproduce RACE score
#11 opened by sugiyama-hiroaki - 1
- 5
transfer learning code
#18 opened by xuy2 - 4
Language Model Generation Example
#1 opened by aykutfirat - 2
Any timeline to release the code to train the LM + finetune on the other 11 tasks?
#13 opened by Franck-Dernoncourt - 0
zip argument error
#14 opened by mehdimashayekhi - 2
What is the meaning of M in the inputs
#12 opened by ioana-blue - 1
- 1
Question about the shape of `X_train`
#9 opened by FrankWork - 1
Supported languages
#2 opened by ErikTromp - 1
Unable to adapt language model
#7 opened by chiayewken - 1
Concatenating context and embeddings?
#4 opened by windweller