RUCAIBox/TextBox

SeqGAN Model with Huge Data

Closed this issue · 1 comments

hmqt commented

Hi,
I am using TextBox tools specially SeqGAN Model with Arabic texts for text generation.

When I upload a large file (train.tgt with 5 Giga size), the SeqGAN model after running gets me:
"
24 Mar 14:25 INFO Loading data from scratch
killed
"
Why does this show me that is there a limitation on data size? Because when I use train.tgt with 2 Mega, the SeqGAN model runs correctly, but its results are poor.

According to your experiences and observations, what are the factors and parameters that I can change in order to improve the generation of Arabic texts?

Thank you :)

This is most likely caused by insufficient memory on your machine. Perhaps a machine with a larger memory can help you.