What is the relationship between ulmo and this project?
jkreft-usgs opened this issue · 7 comments
Ulmo is a project that sounds like it has a similar goal to dataRetrieval. It might be cool to see if this project could contribute a more robust NWIS module to ulmo.
I just learned about this package. I'm not an R user, so I don't use dataRetrieval per say; but I've known about it for a long time and it looks fantastic.
ulmo does have "robust" (*) NWIS access services, including a decent test battery integrated on its Travis CI. ulmo also has been around for a fairly long time (as these things go), so I'd say it's fairly widely used.
I've been the maintainer effectively since Spring 2019. All credit for the ulmo nwis reader, and ulmo in general, go to the original ulmo developers and the many contributors over the years. For the intrepid or CUAHSI fans it's also worth pointing out that NWIS data are also available from ulmo via the CUAHSI HIS/WOF readers.
FYI I was working towards a new release in late January but got seriously derailed. It's all but complete, except for some additional documentation updates, possible deprecation of one of the readers, and other house cleaning. I've been meaning to complete the release ASAP, maybe by next week.
I'd be happy to describe the strengths and weaknesses of ulmo in general as I see them, circa 2020 April. You can get some sense of its soul searching since late 2018 on this closed issue.
It's also worth pointing out another Python NWIS implementation, hydrofunctions. I'm pinging its developer @mroberge. 3 separate NWIS Python implementations is definitely a call for some community discussion and alignment of efforts.
(*) It's only as robust as the bugs that are reported by users and how quickly we can fix them! But this recently closed issue speaks well of the user community, I think -- and thanks to @jkreft-usgs for adding an chiming in officially on the topic.
On another note, I've been interested in adding WQP/WQX functionality to ulmo, but it's never been a priority, unfortunately. Great to see the WQP focus here. I think I see that WQX parsing is not yet within the scope of this package, but FYI this parser is worth looking into. It would take some effort to disentangle it from the aspects that are tightly integrated into the parent package, pyoos (which is effectively deprecated), but still. Though, with progress being made on the WQP API, @jkreft-usgs has discouraged me in the past from diving into full WQX parsing.
Hi, I'm the developer for hydrofunctions. Thanks for pinging me! Hydrofunctions grew out of code I was using to download and process NWIS data.
My goal was to maintain the NWIS funtionality (like keeping the data quality flags) while making it easy for my non-Python students to use (informative error messages; resampling datasets with different frequencies). I've tried to have extensive testing and documentation.
I've seen the R dataRetrieval- it's great! I'm not familiar with this 'dataretrieval' project.
On the Python side, WellApplication uses some code from hydrofunctions and has the ability to parse other data sources. Hydrofunctions gets used as a dependency for tabs.
I'd love to talk about organizing or cooperating in the future. I'm sure there are others out there too. Let me know how I can contribute.
@emiliom @mroberge @jkreft-usgs
Looking for general advice. Or even just pointers to resources that would help me gather info to make a decision.
I used ulmo successfully a year or two ago to retrieve USGS river flow data. But now it doesn't seem very functional any more. (For one thing, I can't get it to install within conda, due to conflicts with other standard packages apparently.)
So it looks like I need to update my code using something other than ulmo. I see hydrofunctions, dataretrieval, and several other similar python packages out there but some are out of date (hydropy)-- and I'm not sure I'm aware of all of them.
In your opinions what currently functional python tools for retrieving flow data from USGS are the most commonly/broadly used and/or most likely to be supported in to the future?
If possible, I would really prefer a package that can be installed through conda-forge instead of by pip.
Thank you in advance for any suggestions.
Yes, @mroberge I have used pip before. (It's not a requirement that I stay with conda only and not use pip, just simply a strong preference. Helps me to more easily update my virtual environment.)
I just installed ulmo by pip, which of course worked fine. I ran my old code, which uses ulmo, and it works fine. So I am all set for the short-term.
Hopefully whatever change is needed so that I can get ulmo in the future using conda-forge without any conda conflicts is easy for you folks to figure out and implement, so that next time I update my conda env I can just include ulmo there instead of using pip.
Thank you both for your tips.
Oops that prior post here was for the thread over at ulmo.
Bottom line is that I'm all set now.
I have learned that I was incorrect to think ulmo is not very functional any more. The problem is that installing it by conda-forge is not working for me. But I installed it by pip, and other than that, it is working fine.
Thanks again.
dataretrieval focuses on USGS water data and is supported directly by USGS. We're happy to see overlap with other projects as they all make USGS data more accessible, which is a good thing. Healthy community-supported packages are a good thing. But so is having better well-supported tools from the data originators.