karel-brinda/Phylign

When many cobs matches reported, batch_align uses an extremal amount of memory & swaps

karel-brinda opened this issue 2 years ago · 3 comments

karel-brinda commented 2 years ago

The query file has 11 GB, without headers 85 MB

karel-brinda commented 2 years ago

Reason:
https://github.com/karel-brinda/mof-search/blob/08c28f8366ad35ffcb79f9953b3668494e47c38a/scripts/batch_align.py#L400
is not really selected, but all that appear in the query file.

karel-brinda commented 2 years ago

We should somehow pass a list of refs in a given batch.

karel-brinda commented 2 years ago

The filtration should be done already here: https://github.com/karel-brinda/mof-search/blob/08c28f8366ad35ffcb79f9953b3668494e47c38a/scripts/batch_align.py#L112