Batch size in training GPT_2
ngocpham97 opened this issue · 1 comments
ngocpham97 commented
I have a question about batch size in train.py. I wonder batch_size =1 meaning 1 token will pass to model or 1024 token. Anyone explain help me ? Thank a lots
yoonsikp commented
It means 1 training example in the batch with a token size of 1024.