Remove calculated variables from CONUS404 zarrs
Opened this issue · 6 comments
Variables to remove: E2, ES2, RH2, SH2
Based on: https://doimspp.sharepoint.com/:w:/r/sites/gs-wma-hytest/_layouts/15/Doc.aspx?sourcedoc=%7B49275EF5-7120-4B12-A3B3-94782ECC8AE0%7D&file=CONUS404%20Zarr%20Derived%20Variables.docx&action=default&mobileredirect=true
For now, remove from metadata so they don't show up, but keep data in zarr store.
Will keep the derived variables PWAT, TK, Z because they are published on RDA. They are only available in the 3D files, so it would be difficult to reproduce if needed.
@pnorton-usgs will edit zmetadata file on Caldera and write some notes about what was removed and how it can be added back to restore functionality of variables.
Amelia will sync this change to S3 and OSN
@amsnyder The zmetadata files for the conus404-raw hourly, daily, and monthly zarr stores have been updated and verified for the onprem datasets.
The updated metadata has been copied onto S3 and the OSN pod.
@amsnyder Recently the conus404-raw hourly, daily, and monthly zarr stores were extended to include time steps through 2022-10-01 (see #369). The derived variables E2, ES2, RH2, and SH2 were not updated in the hourly zarr and the daily and monthly zarr stores were recreated so they no longer have those derived variables at all. I've reopened this issue to keep continuity with our prior decision about keeping the derived variables but removing them from the consolidated metadata. I propose we remove these derived variables entirely from the hourly zarr store. If they are needed at some point in the future we can recreate these variables with our workflows.
@amsnyder I have removed E2, ES2, RH2, and SH2 entirely from the conus404-raw hourly on-prem dataset. Once these variables are also removed from our cloud holdings this issue can be closed (again).