nasa/opera-sds-pcm

[New Feature]: DSWx-S1 Validator Script for OPS Production Audits

Closed this issue · 4 comments

riverma commented

Checked for duplicates

Yes - I've already checked

Alternatives considered

Yes - and alternatives don't suffice

Related problems

The current DSWx-S1 validator script returns only a list of MGRS tile set IDs. For OPS to use this information to audit whether processing is performing correctly and delivering the right products to CMR, this information needs to be manually converted to expected granule IDs that CMR should contain and OPS needs to manually check CMR.

Describe the feature request

The validator script should include an additional feature that takes the list of expected tile set IDs, and returns the list of granule IDs expected to be produced while also checking CMR if they are already delivered.

riverma commented

@LalaP: what information does OPS need to verify DSWx-S1 processing? Please let me know if what I captured above is sufficient.
@hhlee445: we need delivery information of DSWx-S1 at CMR, like collection name, granule ID signature. Could you help provide:

  • A sample query at CMR for DSWx-s1 products
  • A breakdown of the granule ID file name, what the components of the file name are, and which ones can be "predicted" based on input RTC granule metadata + MGRS Tile Set IDs
riverma commented

Update on the second bullet point above:

Looked into it and i don't think there's a direct predictive quality to the DSWx-S1 granule ID signature that will tie back to input RTCs or MGRS Tile Set IDs. I think we'd have to search the DSWx-S1 collection and request matches for the InputGranules field against the input RTCs for a given product we think should exist.

See the Metadata tab for this example granule: https://search.uat.earthdata.nasa.gov/search/granules/granule-details?p=C1261113948-POCLOUD&pg[0][v]=f&pg[0][gsk]=-start_date&g=G1264681475-POCLOUD&q=DSWX&tl=1719277753!3!!

hhlee445 commented

mgrs_set_id maps to mgrs_tiles

example:

MS_112_60 -> mgrs_tiles list ['23VNJ', '23VNK', '23VPJ', '23VPK', '24VUP', '24VUQ', '24VVP', '24VVQ']

  1. OPERA_L3_DSWx-S1_T23VNJ_20240622T091336Z_20240625T030625Z_S1A_30_v0.4
  2. OPERA_L3_DSWx-S1_T23VNK_20240622T091320Z_20240625T030631Z_S1A_30_v0.4
  3. OPERA_L3_DSWx-S1_T23VPJ_20240622T091333Z_20240625T030638Z_S1A_30_v0.4
  4. OPERA_L3_DSWx-S1_T23VPK_20240622T091318Z_20240625T030549Z_S1A_30_v0.4
  5. OPERA_L3_DSWx-S1_T24VUP_20240622T091332Z_20240625T030557Z_S1A_30_v0.4
  6. OPERA_L3_DSWx-S1_T24VUQ_20240622T091318Z_20240625T030604Z_S1A_30_v0.4
  7. OPERA_L3_DSWx-S1_T24VVP_20240622T091329Z_20240625T030611Z_S1A_30_v0.4
  8. OPERA_L3_DSWx-S1_T24VVQ_20240622T091316Z_20240625T030618Z_S1A_30_v0.4

MS_112_59 -> mgrs_tiles list ['23VNL', '23VPL', '23WNM', '23WPM', '24VUR', '24VVR', '24VWR', '24WVS', '24WWS']

  1. OPERA_L3_DSWx-S1_T23VNL_20240622T091306Z_20240625T025330Z_S1A_30_v0.4
  2. OPERA_L3_DSWx-S1_T23VPL_20240622T091305Z_20240625T025402Z_S1A_30_v0.4
  3. OPERA_L3_DSWx-S1_T23WNM_20240622T091258Z_20240625T025341Z_S1A_30_v0.4
  4. OPERA_L3_DSWx-S1_T23WPM_20240622T091258Z_20240625T025320Z_S1A_30_v0.4
  5. OPERA_L3_DSWx-S1_T24VUR_20240622T091303Z_20240625T025413Z_S1A_30_v0.4
  6. OPERA_L3_DSWx-S1_T24VVR_20240622T091302Z_20240625T025351Z_S1A_30_v0.4
  7. OPERA_L3_DSWx-S1_T24VWR_20240622T091259Z_20240625T025444Z_S1A_30_v0.4
  8. OPERA_L3_DSWx-S1_T24WVS_20240622T091259Z_20240625T025425Z_S1A_30_v0.4
  9. OPERA_L3_DSWx-S1_T24WWS_20240622T091259Z_20240625T025435Z_S1A_30_v0.4
  • 👍1
riverma commented

Thank you @hhlee445!