glamod/glamod-ingest

Fix matching of source_id when observation_id includes station_id with hyphen in it

Closed this issue · 2 comments

My code splits on the hyphen, so you are right, I might need to rerun everything.

I currently do this:

primary_id, record_number = x['observation_id'].split('-')[:2]

So, I would convert this as follows:

id = " BR00B6-0360-1-1965-01-01-44-13"
primary_id, record_number = id.split('-')[:2]
primary_id == 'BR00B6'
record_number == '0360'

What is the foolproof method of breaking it up?

  • Does it always end with: --
    -<observed_variable>-<value_significance> ?
  • If so, I can split off the last 5 items first…
  • …then:
    • record_number = last item
    • primary_id = first items joined by '-'

Will that do the job?

A new method of working back on the split would work, the files will always have :

Daily:  <YYYY>-<MM>-<DD>-<observed_variable>-<value_significance>
Sub daily : <YYYY>-<MM>-<DD>-<HH:MM>-<observed_variable>-<value_significance>
MONTHLY : <YYYY>-<MM>-<observed_variable>-<value_significance>

Implemented a fix in: b7c21d1