Allow for local path to IONEX data
Closed this issue · 4 comments
Hi @maaijke,
It'd be really useful to be able to handle local (or other exotic server) versions of IONEX data. This is often required on HPCs where internet access is locked down - especially for FTP connections! Thankfully, urllib
can already handle this with e.g. server=file://path/to/the/data
.
Along side this, it'd be handy to be able to pass a formatter for the file naming convention used. This is currently somewhat hard-coded in the get_urllib_IONEXfile
function. My suggestion would be to have an optional formatter
argument that is called to appropriately format the URL strings, with a lookup table of known formatters e.g.
KNOWN_FORMATTERS = {
"ftp.aiub.unibe.ch":
lambda server, year, dayofyear, prefix, yy:
f"{server}/CODE/{year:4d}/{prefix.upper()}{dayofyear:03d}0.{yy:02d}I.Z",
"cddis.gsfc.nasa.gov":
lambda server, year, dayofyear, prefix, yy:
f"{server}/gnss/products/ionex/{year:4d}/{dayofyear:03d}/{prefix}{dayofyear:03d}0.{yy:02d}i.Z",
"igsiono.uwm.edu.pl":
lambda server, year, dayofyear, prefix, yy:
f"{server}/data/ilt/{year:4d}/igrg{dayofyear:03d}0.{yy:02d}i",
}
@tjgalvin and myself have been playing with implementing these changes and along the way we also found that, due to the speed of a local cache, some race conditions can occur. We've put in some checks to catch these.
Would you be open to a PR that brings in these changes? All being well, the current behaviour should be completely preserved.
Hi @maaijke - would you be interest this at all?
Hi Alec, I think something went wrong with my earlier reply, sorr for that. YEs I am interested, especially in the formatter!
Local IONEX files is already supported, but if you have a better way of doing that I am happy to have a look at that
Ok cool - I'd be curious to see what is currently supported! I'll open a draft PR so you can inspect the modifications I've been using locally. I'm keen to see what you think :)
Hey @maaijke - thanks for the merge! Just wondering, will you push a new release of RMextract along with this?