Pin1yin1/pin1yin1

Import scripts use a lot of RAM

Opened this issue · 2 comments

For some reason, maybe memory leaks, the import scripts require several hundred megabytes of RAM, which makes them hard to run successfully on a small VM.

With the latest version of the data files, importing uses about 300M.

Wowzers, that's special :)

I clocked the following (approx overall RSS):

  • import_bopomofo 86MB
  • import_cedict 115MB
  • import_zi 305MB
  • record_definition_characters 482MB

So, this is well confirmed.