Memory overflow

[pedrosenna]

Hello everyone,

is there any way to reduce the memory allocation of flexsdm? Its simply impossible for me to run some functions, like correct_colinvar() and sdm_predict() since it eats up all memory of my notebook (i7-7500, 16 GB RAM, Ubuntu 22) and i'm using 13 cropped layers of 5400 x 6000 and ~1 km resolution. My notebook is not the best for the task but i feel this memory issue can make some people simply drop the package, mostly students and people who cannot afford for better setups.

For example, raster and terra packages create temp files on a temp dir and allows the user to set memory fraction manually when using rasterOptions(memfrac= value) or terraOptions(memfrac= value). It also can process files into chunks if needed as well.

Sincerely,

Pedro Bittencourt

[sjevelazco]

Hi Pedro Thank you for reporting this issue. I do not have a solution in mind right now. I am copying the other package authors; we can think of a solution soon. I will let you know if we improve these functions. Best El jue, 23 mar 2023 a las 0:31, Pedro Senna Bittencourt (< ***@***.***>) escribió:

…

Hello everyone, is there any way to reduce the memory allocation of flexsdm? Its simply impossible for me to run some functions, like correct_colinvar() and sdm_predict() since it eats up all memory of my notebook (i7-7500, 16 GB RAM, Ubuntu 22) and i'm using 13 cropped layers of 5400 x 6000 and ~1 km resolution. My notebook is not the best for the task but i feel this memory issue can make some people simply drop the package, mostly students and people who cannot afford for better setups. For example, raster and terra packages create temp files on a temp dir and allows the user to set memory fraction manually when using rasterOptions(memfrac= value) or terraOptions(memfrac= value). It also can process files into chunks if needed as well. Sincerely, Pedro Bittencourt — Reply to this email directly, view it on GitHub <#327>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGW3YWD4QH6G5UDRQ7WSENLW5O7XXANCNFSM6AAAAAAWEUPQFQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[sjevelazco]

Dear Pedro

We implemented in the flexsdm v.1.3.3 the possibility of sample rasters to reduce machine memory and speed up the process in correct_colinvar function and predict model in chunks to reduce machine memory in sdm_predict.

Thank you for these suggestions!

[pedrosenna]

Dear Pedro

We implemented in the flexsdm v.1.3.3 the possibility of sample rasters to reduce machine memory and speed up the process in correct_colinvar function and predict model in chunks to reduce machine memory in sdm_predict.

Thank you for these suggestions!

That's amazing! Thanks a lot!