Fix timing correction to work with most recent data
Closed this issue · 2 comments
jrs65 commented
At the moment the ApplyTimingCorrection
task fails when running on more recent data that doesn't need it at all. We need to fix it such that it simply doesn't apply anything on this data.
Seth suggested that the best way to do this would be to use a dataflag to indicate the range of time over which a timing correction should in theory be applied, although it might be easier to just hard code the switch over data in the code.
anjakefala commented
Creating a dataflag using cdf:
https://bao.chimenet.ca/wiki/index.php/Cdf
Sample code that excludes days based on a dataflag:
base = Path("/path/to/files/")
available_lsds = np.array(sorted([int(path.stem.split("_")[-1]) for path in base.glob("*/sstream_lsd_*.h5")]))
dates = ctime.unix_to_datetime(ephemeris.csd_to_unix(np.array(available_lsds)))
core.connect()
bad_flags = [
'needs_timing_correction',
]
flags = df.DataFlag.select().join(df.DataFlagType).where(df.DataFlagType.name << bad_flags)
flag_time_spans = [(f.type.name, f.start_time, f.finish_time) for f in flags]
lsd_time_range = ephemeris.csd_to_unix(available_lsds[:, np.newaxis] + np.array([[0, 1]]))
anjakefala commented
id: None
type: needs_timing_correction
start: 2018-10-11 14:00:00+00:00
finish: 2020-03-04 01:13:01+00:00
metadata:
{instrument: chime, user: Anja Boskovic}