LiberTEM/LiberTEM-live

Merlin SIM fails w/ many files and `memfd`

sk1p opened this issue · 0 comments

sk1p commented

Debugging with @matbryan52: if the Merlin dataset has many files (thousands), it will exhaust the number of file descriptors available by default. Increasing the limit can then cause another issue, as we are using select to check for connection abortions, and select only supports file descriptors in [0, 1024).

Why we are using select: Windows. See commit ec9953c

Possible fix is to keep using select on Windows, and switch to the previous implementation using poll or epoll for Linux, which works properly for "high" file descriptor numbers.