solve the performance degradation when computing the latitude weights
Opened this issue · 0 comments
iluise commented
When using the weighted_mse
loss we see a performance degradation due to the computation of the latitude weights in multifield_data_sampler.py
.
Tasks: speed up weight computation so to avoid the hiccups in the performance.
epoch: 0 [1/91 (1%)] Loss: 1.17013 : 0.91218 :: 0.56689 (1.63 s/sec)
epoch: 0 [2/91 (2%)] Loss: 1.25394 : 1.05042 :: 0.56396 (87.65 s/sec)
epoch: 0 [3/91 (3%)] Loss: 1.29323 : 1.13104 :: 0.56055 (35.11 s/sec)
epoch: 0 [4/91 (4%)] Loss: 1.10516 : 0.81147 :: 0.55713 (16.48 s/sec)
epoch: 0 [5/91 (5%)] Loss: 1.16656 : 0.92310 :: 0.55420 (87.29 s/sec)
epoch: 0 [6/91 (7%)] Loss: 1.31812 : 1.18307 :: 0.55176 (3.58 s/sec)
epoch: 0 [7/91 (8%)] Loss: 1.21557 : 1.01287 :: 0.54932 (89.66 s/sec)
epoch: 0 [8/91 (9%)] Loss: 1.11991 : 0.83303 :: 0.54639 (84.81 s/sec)
epoch: 0 [9/91 (10%)] Loss: 1.19671 : 0.98798 :: 0.54443 (36.95 s/sec)
epoch: 0 [10/91 (11%)] Loss: 1.17417 : 0.93769 :: 0.54199 (15.76 s/sec)
epoch: 0 [11/91 (12%)] Loss: 1.12489 : 0.87167 :: 0.53955 (84.16 s/sec)
epoch: 0 [12/91 (13%)] Loss: 1.15331 : 0.90734 :: 0.53857 (3.78 s/sec)