UCL-CCS/EasyVVUQ

Excessive amounts of memory are sometimes used by the PCE analysis phase

Closed this issue · 1 comments

Excessive amounts of memory are sometimes used by the PCE analysis phase

I have traced this down to the production of two outputs which might not be needed: the correlation_matrices (which in one of my cases accounts for 1.490 GB) and the output_distributions (which accounts for 9.780 GB for the case mentioned).

I am in the process of preparing some changes to the code so that the calculation of these outputs can be disabled.

Preliminary results are

                      | default Corr & Dist  | no Corr & Dist | ratio |
----------------------|----------------------|----------------|-------|
Time for analysis (s) |     20600            |    2617.06     | 0.127 |
user time (s)         |     19330.81         |    2563.83     | 0.133 |
sys time (s)          |      1834.21         |      86.13     | 0.047 |
elapsed time (s)      |     20632.66         |    2626.67     | 0.127 |
memory usage (bytes)  |     360 269 376      |    6 823 864   | 0.019 |

Memory usage drops to 2% and the elapsed time to 12.7%.

A pull request should come within the next few days, after some more testing.

I think we can close this ...