jadore801120/attention-is-all-you-need-pytorch

train big data(8G)

JoeCoding opened this issue · 0 comments

use preprocess.py to deal with big data(8G) to product 12G .pkl file. why it cost about 150G memory, when loading this .pkl file.