bpbond/cosore

Soil moisture in wrong units for some datasets

Opened this issue · 3 comments

arthur-e commented

According to the 2020 paper in Global Change Biology, soil moisture in the "data" tables should be in volumetric m^3/m^3 but this seems to vary by dataset. "d20200114_CARBONE_BURNS", "d20190526_RAICH_survey", and "d20200328_UEYAMA_HOKUROKU" are three (there may be more) that seem to have soil moisture in percent (%) units.

> csr_table('data', 'd20190526_RAICH_survey')
d20190526_RAICH_survey	Reading standardized data
# A tibble: 4,858 x 10
   CSR_PORT CSR_TIMESTAMP_BEGIN CSR_TIMESTAMP_END   CSR_FLUX_CO2 CSR_DRY_CO2 CSR_RH CSR_SM5 CSR_T5 CSR_TAIR_AMB
      <int> <dttm>              <dttm>                     <dbl>       <dbl>  <dbl>   <dbl>  <dbl>        <dbl>
 1        3 2004-09-29 10:08:30 2004-09-29 10:10:00         7.92        423    78.1    24.3   25           26.6
 2        3 2004-09-29 10:14:30 2004-09-29 10:16:00         6.94        399    74.3    37.8   24.7         26.6
 3        3 2004-09-29 10:18:30 2004-09-29 10:20:00         7.67        417.   77.2    36.9   25           26.7

For now, I'm going to assume datasets with a maximum soil moisture value < 1 are in m^3/m^3 units and those with a max. > 1 are in percent units.

arthur-e commented

Also, at least one dataset has negative soil moisture values!

> csr_table('data', 'd20200212_KAYE_UNW')
d20200212_KAYE_UNW	Reading standardized data
# A tibble: 14,707 x 7
   CSR_PORT CSR_TIMESTAMP_BEGIN CSR_TIMESTAMP_END   CSR_FLUX_CO2 CSR_SM5 CSR_T5 CSR_DATASET       
      <int> <dttm>              <dttm>                     <dbl>   <dbl>  <dbl> <chr>             
 1        0 2015-07-13 23:51:30 2015-07-13 23:53:00         5.6   -0.840   19.0 d20200212_KAYE_UNW
 2        0 2015-07-14 00:51:30 2015-07-14 00:53:00         5.77  -0.837   18.9 d20200212_KAYE_UNW
bpbond commented

Thanks @arthur-e ! Great to identify. I will investigate about the fraction vs. percentage thing—ugh, tests should catch this.

The negative issue, hmm, separate but good to know. Thanks.

arthur-e commented

I hate to pile on here, but there are a couple of other datasets to note. Below are percentiles for the CSR_SM5 field (5-cm soil moisture). The TP39 dataset has the correct units (m3 m-3) but the other two do not seem to be correct, i.e., there is too little variation around (and above) 1.0 for these to be in percent units.

Dataset Port Flux 1% 10% 50% 90% 99%
d20200417_ARAIN_TP39 1 Rs 0.06 0.06 0.10 0.15 0.19
d20200417_ARAIN_TP39 2 Rs 0.06 0.06 0.11 0.15 0.20
d20200417_ARAIN_TP39 3 Rh 0.06 0.06 0.11 0.15 0.21
d20200417_ARAIN_TP39 4 Rs 0.06 0.06 0.10 0.15 0.20
d20200417_ARAIN_TP39 5 Rs 0.06 0.06 0.10 0.14 0.19
d20200417_ARAIN_TP39 6 Rs 0.06 0.06 0.10 0.14 0.19
d20200417_ARAIN_TP39 7 Rs 0.06 0.06 0.10 0.14 0.18
d20200417_ARAIN_TP39 8 Rh 0.06 0.06 0.09 0.14 0.18
d20200417_ARAIN_TP74 1 Rs 1.09 1.15 1.21 1.45 1.63
d20200417_ARAIN_TP74 2 Rs 1.19 1.21 1.32 1.56 1.65
d20200417_ARAIN_TP74 3 Rs 1.19 1.21 1.38 1.59 1.71
d20200417_ARAIN_TP74 4 Rs 1.18 1.25 1.48 1.59 1.66
d20200417_ARAIN_TPD 1 Rs 0.19 0.76 0.89 1.11 2.04
d20200417_ARAIN_TPD 2 Rs 0.78 0.88 1.05 1.15 1.50
d20200417_ARAIN_TPD 3 Rs 0.03 0.17 0.91 1.12 3.47
d20200417_ARAIN_TPD 4 Rs 0.77 0.81 1.03 1.11 1.18
d20200417_ARAIN_TPD 5 Rs 0.73 0.75 1.00 1.09 1.15

I might reach out to the authors and see if they can clarify. I assume they would have the option of uploading correcting data to COSORE.