About the memory space and script
gdmdxl opened this issue · 2 comments
Hello!
Maybe it's a stupid question, but I need to solve it. My hard drive space is limited. During correction/Creating overlap store correction/daisheng.ovlStore, if one of buckets is completed(I have 181 processes), can the corresponding *.ovb file be deleted to solve the space shortage problem?
Here are part of the output files:
__-- BEGIN CORRECTION
--
-- Creating overlap store correction/daisheng.ovlStore using:
-- 191 buckets
-- 665 slices
-- using at most 29 GB memory each
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'ovB' concurrent execution on Sat Mar 2 11:13:45 2024 with 5822.922 GB free disk space (181 processes; 36 concurrently)_
Thank you very much!
Yes, generally you can do this. The FAQ details some ways to reduce space usage: https://canu.readthedocs.io/en/latest/faq.html#my-assembly-is-running-out-of-space-is-too-slow and what is safe to delete. The option purgeOverlaps=agressive
(https://canu.readthedocs.io/en/latest/parameter-reference.html?highlight=purgeOverlaps#cleanup-options) will do this as well but at the risk of removing some files early so if a job fails you might need to re-run some computational steps.
Thank you very much for your patient answer, which has solved my problem very well.