Feature request: Do not crash when aligning larger files when Local or Global algoritm is selected
MatejKafka opened this issue · 3 comments
With files larger than ~1MB, biodiff crashes when using the Local or Global algorithm, as it runs out of memory. I'd prefer if it aborted the align with an error message (directing the user to use the Blockwise algorithm, or even automatically switching), and ideally allowed the user to continue with other operations.
The crash is kinda hard to debug at least on Windows, because biodiff aborts with a generic error code, and the TUI typically hides the error message. This is also made a bit more annoying by biodiff switching the console to an alternate screen buffer (https://stackoverflow.com/a/39190527), and when it crashes, it does not return to the original buffer and user has to rerun biodiff and exit to return.
Yeah, it's probably running out of memory because these algorithms do need a lot of memory. There's a newer algorithm, biwfa, which needs significantly less memory.
I've actually tried to use it in the wfa2 branch some time ago if you would like to try that out, but note that the library does not support local alignment for biwfa, so only block align and global will work (it falls back to rustbio for other cases).
I'm still not sure how to integrate all of it into a settings window that is not too overwhelming, but ideally I would somehow allow the user to choose from multiple backends or something.
But yes ideally I would somehow catch the error to show a more friendly error message, but for example on linux the program will overcommit memory just fine and then spontaneously crash when using too much memory without a way to catch that. I don't think rustbio for example exposes an interface for predicting the allocated memory.
Current main now puts out a warning message (which you can choose to ignore) if alignment could lead to high memory usage.