ropensci/FedData

Use arcgislayers as backend for ESRI's AGOL

Closed this issue · 12 comments

kbvernon commented

Hi Kyle, I think I mentioned this to you before, but I want to suggest adopting {arcgislayers} as a backend for interacting with ESRI's AGOL. That package is going through some additional testing, but I believe it should reach a stable version in the next couple of weeks or month.

Benefits:

  1. {arcgislayers} is officially supported by ESRI. That's maybe not a great argument... but!
  2. It would remove a ton of code from {FedData} that you would no longer have to maintain.
  3. It would make it easier to introduce some functionality that we've talked about before, like query support (#68) and attribute/column selection.
  4. And, it can be done without introducing breaking changes.

Costs:

  1. Maybe AWS is better? (I have almost no experience with AWS and am not sure this is entirely true, so take it for what it is.)
  2. {arcgislayers} is built around the idea of interacting with AGOL (upload, download, query, retrieve metadata, etc), so there's a risk of mission creep.

Just to give an example of the trade-off here, {arcgislayers} gives you the power to query AGOL and return a table without spatial data, which is nice for exploring data before figuring out what features to download. It also allows you to specify what attribute data you want to get. But right now, {FedData} always returns all of the spatial and attribute data. My own feeling is that the benefits of filtering rows and selecting columns outweigh the potential risks, but a hardline should definitely be drawn there.

Also, get_nhd() will require some special treatment since it returns a list. My guess is additional functions for retrieving individual components of NHD, so that uber function can be left alone.

What do you think?

If it helps, this is something I would be more than willing to invest some time. Least I can do since this package factors so heavily in all of my research.

bocinsky commented
  • ❤️1
kbvernon commented

Thanks, Kyle! I'm going to wait till they release a stable version before I start working on this (probably mid-February), then I can share a PR.

Cheers.

bocinsky commented
JosiahParry commented

If you all end up taking this on, please let me know if I can be of assistance! I'm sure y'all will run into some "nice to haves" and if you do, please let me know. Additionally, if there is any progress made on this before March 10th-ish, I'd love to talk about it at Esri DevSummit

  • ❤️1
bocinsky commented

Will do. Thanks @JosiahParry!

bocinsky commented

Hi y'all. I've begun the implementation of this, first by simply using arcgislayers without changes to the FedData api. So, query support requested in (#68) is not yet implemented. But it works well, and I'm pleased to rely on an official ESRI package for accessing ArcGIS layers.

That being said, we can't release this update to CRAN until arcgislayers is on CRAN. I didn't put two and two together about this until I'd already implemented it. @JosiahParry, any notion on if/when a CRAN release of arcgislayers is likely to happen?

  • ❤️2
JosiahParry commented

Yup! Totally fair point. :)))
@bocinsky The goal is a CRAN release within the next few weeks. Before March 12! I need to sniff out as many bugs as possible and fix them before then. So...if you have any that you've encountered please do let me know. I'll keep you posted on this. Sorry for the headache!

bocinsky commented
JosiahParry commented

arcgisutils 0.2.0 is released on CRAN

arcgislayers is waiting for review: https://cran.r-project.org/incoming/newbies/

  • 🎉1
bocinsky commented
  • ❤️2
bocinsky commented

Great news! Installed and running checks now for FedData resubmission to CRAN