kpu/kenlm

Cannot allocate memory Failed to allocate when using build_binary

amitliron opened this issue · 0 comments

I have a arpa file which I created by the following command:

./lmplz -o 4 -S 1G <tmp_100M.txt >100m.arpa

Now I want to convert this arpa file to binary file:
./build_binary 100m.arpa 100m.bin

And I'm getting error:

mmap.cc:225 in void util::HugeMalloc(std::size_t, bool, util::scoped_memory&) threw ErrnoException because `!to.get()'.
Cannot allocate memory Failed to allocate 106122412848 bytes Byte: 80
ERROR

I tried to add -S parameter:
./build_binary -S 1G 100m.arpa 100m.bin
and I got the same error.

  1. How can I convert to binary file ?
  2. Why I'm getting this error ?