flow directions and their representations
emontgomery-usgs opened this issue · 12 comments
Rich's answer to the question about how to display the wind, wave, current direction variables in the portal is "as they are".
This means than wind and waves should be "from_direction" and current (CD_310) should be "to_direction". In files where CD_310 already exists, it will be correct ("to") but where you compute it, I believe it will be "from", but should be "to".
I looked at FI14-989 (https://cmgdata.usgsportals.net/#module-metadata/590b5a22-b61e-11e4-b2db-00265529168c/6878f136-b61e-11e4-a8f5-00265529168c) and it looks like the portal only displays speed for currents (no direction), so the fact that the direction is likely to be 180 off what's expected isn't exposed.
- note the description of the CF standard_name "direction_of_sea_water_velocity" doesn't clearly state that it's a "to_direction", but Rich and I both expect it follows oceanographic convention where currents flow "to".
@emontgomery-usgs , looking into this.
So we have established that:
- wind (and waves) should have values consistent with
standard_name=wind_from_direction(e.g. 0 degrees = wind from north, 90 degrees = wind from east) - ocean current should have values consistent with
standard_name=sea_water_to_direction(e.g. 0 degrees = flow toward north, 90 degrees = flow toward east)
FWIW, I checked the standard_name list and both sea_water_from_direction and sea_water_to_direction are listed as options. Perhaps they are new?
I checked a sample location in West Falmouth and the current looks okay (90 degrees is current toward east):
http://erddap.axds.co/erddap/tabledap/56455.graph?eastward_sea_water_velocity_8871aqd_a%2Csea_water_velocity_to_direction_8871aqd_a%2C&time%3E=2010-09-02T00%3A00%3A00Z&time%3C=2010-09-09T00%3A00%3A00Z&.draw=markers&.marker=5%7C5&.color=0x000000&.colorBar=%7C%7C%7C%7C%7C&.bgColor=0xffccccff
@kwilcox, one small thing to fix: we have standard_name=sea_water_velocity_to_direction but that is not a valid standard_name. It should be standard_name=sea_water_to_direction.
@rsignell-usgs @kwilcox
The root of the problem is applying normalize_vectors() to all direction variables in the process of generating the CF version. I disabled this function in the _noregex version to allow the CF directions to retain the same from/to that they have in EPIC. Not sure what this will do downstream if ingested to the portal- it assumes all direction data points the same way, so I think we need some changes to allow wind and wave directions to be displayed as "from" and currents to be displayed as "to".
@emontgomery-usgs can you please point to an example in the portal where something is going in the wrong direction?
@rsignell-usgs Sure- the issue is in "direction" variables not east and north. example to follow.
I am perplexed about what your ERDDAP example shows. When I compare the east data in the actual source files, the only difference is a factor of 100 to convert units, and when I plot u_epic vs u_cf, I get nice 1:1 line, not the wiggles your ERDDAP plot shows... I am not concerned about the CF and portal east and north, the problem is how the (mostly computed for CF) direction data is oriented.
@rsignell-usgs @kwilcox - maybe this is partially solved in the new portal, since the direction data is not shown. at https://cmgdata.usgsportals.net/#metadata/56459/station, what is displayed under "currents" is just speed, and I didn't find direction displayed.
But if you delve a bit deeper looking for which sensor the data came from, you get to https://cmgdata.usgsportals.net/#metadata/56459/station/15/sensor, and the direction rose is displayed as a "to" direction. This is current data from a TRDI workhorse ADCP, so "to" direction is correct. The source data is http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/CF-1.6/FI14/9981wh-a.nc.html, and in CD_310, there's standard_name: direction_of_sea_water_velocity, which should be changed to "sea_water_to_direction"
We don't ingest wave data now, but in testing the _regex version of collect.py, I ran into the problems with what happens when waves data is converted to CF. From from tests using the same instrument (9881) the CF version I generate has wave_direction also converted to a "to" direction. http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/tst_wvsa/10631whVp-cal_z2.nc.html. In the wvdir variable there are these attributes, and the data is rotated 180:
standard_name: sea_surface_wave_to_direction
NOTE: Direction FROM which waves are propagating, measured clockwise from north
Not what we want for waves (or wind data, but I haven't looked at that).
The example is not in the portal now, but will be if when we ingest the waves files. I think we need to remove or modify normalize_vectors() prior to ingesting waves data, and make sure the portal won't try to display wave directions as "to" directions.
I found an example from the Met Buoy in the same experiment- it correctly has wind speed displayed as "from".
https://cmgdata.usgsportals.net/#metadata/56422/station
If you want to try to ingest sample waves files of several types, the files here are the outputs of my most recent run of collect_noregex.py with normalize_vectors() disabled. It would be interesting to see how these display! https://gamone.whoi.edu/thredds/silt/usgs/Projects/stellwagen/Data_tests/tst_wvsb/catalog.html
thanks!
TL;DR, nothing that needs to be done at this point unless we want to experiment with wave data?
@rsignell-usgs @kwilcox, I made the standard_name for CD_310 to sea_water_to_direction (mentioned earlier in the thread) in my local version of epic2cf.
@emontgomery-usgs, okay. Is there any action needed on this by @kwilcox ?
@rsignell-usgs, In order to move forward using the correct standard names, my version of epic2cf needs to be pulled into the axiom version. The way I got set-up does not allow for a pull request.
- Then you, Dan, I and whoever else should look the test CF waves files and decide if it's what we want.
- Then we should check with Kyle about how they'd be displayed
- Then we'd ask for ingest (since I don't think auto-harvesting is enabled yet).
@emontgomery-usgs Now that epic2cf is updated we can move on with the bullets!