marbl/canu

About the memory space and script

gdmdxl opened this issue · 2 comments

Hello!

Maybe it's a stupid question, but I need to solve it. My hard drive space is limited. During correction/Creating overlap store correction/daisheng.ovlStore, if one of buckets is completed(I have 181 processes), can the corresponding *.ovb file be deleted to solve the space shortage problem?

Here are part of the output files:

__-- BEGIN CORRECTION
--
-- Creating overlap store correction/daisheng.ovlStore using:
--    191 buckets
--    665 slices
--        using at most 29 GB memory each
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'ovB' concurrent execution on Sat Mar  2 11:13:45 2024 with 5822.922 GB free disk space (181 processes; 36 concurrently)_

Thank you very much!

Yes, generally you can do this. The FAQ details some ways to reduce space usage: https://canu.readthedocs.io/en/latest/faq.html#my-assembly-is-running-out-of-space-is-too-slow and what is safe to delete. The option purgeOverlaps=agressive (https://canu.readthedocs.io/en/latest/parameter-reference.html?highlight=purgeOverlaps#cleanup-options) will do this as well but at the risk of removing some files early so if a job fails you might need to re-run some computational steps.

Thank you very much for your patient answer, which has solved my problem very well.