glamod/glamod-nuim

Issue with date_time` column in cdm converted files

Closed this issue · 5 comments

sjnoone commented

A handful of files have an issue with the date_time not formatting correctly .So the problem appears to be the missing time in the date_time column.

Bad file:

observation_id|report_type|date_time|date_time_meaning|latitude|longitude|observation_height_above_station_surface|observed_variable|units|observation_value|value_significance|observation_duration|platform_type|station_type|primary_station_id|station_name|quality_flag|data_policy_licence|source_id
YMM00041480-2-2010-07-57-12|0|2010-07-12+00|1|12.833|45.033|2|57|32|100450|12|0||1|YMM00041480|ADEN|0|0|286
YMM00041480-2-2010-07-58-12|0|2010-07-12+00|1|12.833|45.033|2|58|32|100480|12|0||1|YMM00041480|ADEN|0|0|286

Good:
observation_id|report_type|date_time|date_time_meaning|latitude|longitude|observation_height_above_station_surface|observed_variable|units|observation_value|value_significance|observation_duration|platform_type|station_type|primary_station_id|station_name|quality_flag|data_policy_licence|source_id
YMM00041416-1-1949-01-30-12:00-36-12|0|1949-01-30 12:00:00+00|1|15.367|42.583|2|36|5|293.75|12|0||1|YMM00041416|KAMARAN|0|1|220
YMM00041416-1-1949-02-06-12:00-36-12|0|1949-02-06 12:00:00+00|1|15.367|42.583|2|36|5|293.15|12|0||1|YMM00041416|KAMARAN|0|1|220
YMM00041416-1-1949-02-08-12:00-36-12|0|1949-02-08 12:00:00+00|1|15.367|42.583|2|36|5|293.75|12|0||1|YMM00041416|KAMARAN|0|1|220

rjhd2 commented

Are we sure that this hasn't already been fixed by this change a4882fd and all that would need doing is to re-run the affected files?

We looked for 00:00 entries that failed, rather than 12:00 ones and fixed for Lite and Obs conversions. Header and QC should inherit....

sjnoone commented

Ah OK that';s fixed then , I'll do a test r-run later on a file and check.

  • 👍1
rjhd2 commented

Ah OK that';s fixed then , I'll do a test r-run later on a file and check.

Worth checking first - happy to look into it again if there's still problems.

sjnoone commented

Ok I will check and make sure it is fixed.

sjnoone commented

yes, I had done a rerun last month and its all good .

observation_id|report_type|date_time|date_time_meaning|latitude|longitude|observation_height_above_station_surface|observed_variable|units|observation_value|value_significance|observation_duration|platform_type|station_type|primary_station_id|station_name|quality_flag|data_policy_licence|source_id
YMM00041480-2-2010-07-12-00:00-57-12|0|2010-07-12 00:00:00+00|1|12.833|45.033|2|57|32|100450|12|0||1|YMM00041480|ADEN|0|0|286
YMM00041480-2-2010-07-12-00:00-58-12|0|2010-07-12 00:00:00+00|1|12.833|45.033|2|58|32|100480|12|0||1|YMM00041480|ADEN|0|0|286

  • 👍1