Minimum Data Requirement Guidelines (minNumObs, minNumUncen)

[elp94]

Hi there,

I've been struggling to find the minimum data requirements for using WRTDS. I've found documentation that indicate WRTDS works best with 100+ observations, and that <50 observations will be in the "dangerous range" for reliability. Would 60 observations be an acceptable cut-off for minimum sample size?

I've also been unable to find any documentation for the maximum number of allowable censored samples for WRTDS (or the minimum number of uncensored samples). The default values for minNumObs and minNumUncen in modelEstimation suggest that WRTDS could run with 50% censored values. Is there any insight to how reliable the regression would be for a dataset that is more than 50% censored? And do you have any recommendations for the maximum percent of censored data that is allowable?

Any guidance or insight that you could provide would be very much appreciated!! Thank you!

[rmhirsch49]

The minimums depend on what you want to do. If it is simply to get an overall estimate of average flux or average concentrations then 50 samples would be ok. But, if you are interested in trends it is preferable that you be at more than 100 samples. For the analysis of trends you can use the tools in EGRETci to see whether your estimates of trend are really meaningful. If the trend was very strong, then you might be able to get a significant result with even somewhat fewer than 100. This would be the kind of thing you might expect to see when there is some major point source that has had substantial improvement over the period of record, but where non-point sources are the main driver of water quality, it would be pretty unusual to get a highly significant result with 100 samples. We don’t have any really specific guidance on the amount of uncensored data. I would certainly say that if you were interested in trends and you had only 100 total samples and 50 or more of them were censored I would say that it is unlikely that you will get a very meaningful result. But, here again, the tools in EGRETci can help you figure that out. I think that the important point is that there are no hard and fast rules on absolute minimums (although you have to adjust minNumObs and minNumUncen to get it to run in some cases) but if all you have is 60 samples, you may want to go with a simpler approach (e.g. Seasonal Kendall for trends). But, if you are working with several sites and/or several constituents and many of them do have more plentiful data (> 100 values) then it might make sense to use WRTDS on all data sets, recognizing that a convincing trend result is unlikely with a small amount of data. I hope that helps. Bob Hirsch From: elp94 <notifications@github.com> Reply-To: USGS-R/EGRET <reply@reply.github.com> Date: Tuesday, February 9, 2021 at 5:52 PM To: USGS-R/EGRET <EGRET@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: [EXTERNAL] [USGS-R/EGRET] Minimum Data Requirement Guidelines (minNumObs, minNumUncen) (#301) This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding. Hi there, I've been struggling to find the minimum data requirements for using WRTDS. I've found documentation that indicate WRTDS works best with 100+ observations, and that <50 observations will be in the "dangerous range" for reliability. Would 60 observations be an acceptable cut-off for minimum sample size? I've also been unable to find any documentation for the maximum number of allowable censored samples for WRTDS (or the minimum number of uncensored samples). The default values for minNumObs and minNumUncen in modelEstimation suggest that WRTDS could run with 50% censored values. Is there any insight to how reliable the regression would be for a dataset that is more than 50% censored? And do you have any recommendations for the maximum percent of censored data that is allowable? Any guidance or insight that you could provide would be very much appreciated!! Thank you! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUSGS-R%2FEGRET%2Fissues%2F301&data=04%7C01%7Crhirsch%40usgs.gov%7C4ac370e9743b45df5cff08d8cd4d4ea0%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637485079201996405%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=H%2FT7dH4vo9sNINCxNwuPIjfLwL%2BG0N2ENE2jYEiytAY%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAI53UI6ZJEZOCJIDO6LNHTS6G37JANCNFSM4XL3SAJA&data=04%7C01%7Crhirsch%40usgs.gov%7C4ac370e9743b45df5cff08d8cd4d4ea0%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637485079201996405%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=18kBTFFpnDUq3CHL6bEke02VA5ksFkJCbb3G%2FGt227M%3D&reserved=0>.

[elp94]

Thank you so much for the response, Bob! It was really helpful and informative, especially for a newbie like me. I appreciate it!