glample/tagger

How to train large quantities of data

Closed this issue · 2 comments

hi,I would like to train more named entities, via more data, about 10 types of entities, 50M of data, but will crash whether through GPU or CPU memory. Is there any solution?

Hi,

50M of data is small, this should not crash. What is the problem you get?

@glample Thank you for your reply. You are right. I was having a problem with the sentence. I treated all the input as an entire sentence. Now that I have solved the problem, I have been training normally and I will report the results of the training later.