algbio/ggcat

code: 24, kind: Uncategorized, message: "Too many open files"

jnalanko opened this issue · 2 comments

jnalanko commented

Hello, I'm trying to run the code on the attached file (file extension .txt because .fna is not allowed on Github). I'm getting the following error:

niklas@phoenix:~/code/ggcat$ ggcat build -k 31 ../Themisto/example_input/coli3.fna -o coli3
Allocator initialized: mem: 2 GiB chunks: 8192 log2: 18
Using m: 12 with k: 31
Started phase: reads bucketing prev stats: 
Temp buckets files size: 26.64 MiB
Finished phase: reads bucketing. phase duration: 373.73ms gtime: 373.80ms
Started phase: kmers merge prev stats: 
Finished phase: kmers merge. phase duration: 452.88ms gtime: 826.68ms
Started phase: hashes sorting prev stats: 
Thread panicked at location: libs/parallel-processor-rs/src/memory_fs/file/internal.rs:248:26
Error message: called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }
Backtrace:    0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: <unknown>
  15: <unknown>
  16: <unknown>
  17: <unknown>
  18: <unknown>
  19: <unknown>
  20: <unknown>

The ulimit for the number of open files is set to unlimited:

niklas@phoenix:~/code/ggcat$ ulimit
unlimited

coli3.txt

Guilucand commented

Hi Jarno,
on my machine processing the file requires a file ulimit of about 5150.
I already set the soft ulimit in the code to match the hard ulimit of the system,
what's the output of ulimit -Hn?

jnalanko commented

I see, my bad. The output of ulimit -Hn is 4096.

  • 👍1