reference file schemas and crds selectors
Opened this issue · 3 comments
As discussed: #256
Reference file schemas place restrictions on selectors that match keywords in science files. These restrictions prevent the use of more complicated selectors (like BETWEEN 0 50
for a keyword that is typically an integer
).
Based on the described selectors here (I'm not sure if these are the same for jwst):
https://hst-crds.stsci.edu/static/users_guide/rmap_syntax.html?highlight=between#selectors
Inclusion of a string type (or perhaps removal of the type constraint) should allow for most of the selectors (except for tuple selectors). I'm not sure how the tuple selectors are encoded in the fits header for reference files.
I'm not sure if there is some filtering between crds selectors and keywords (and perhaps this BETWEEN
selector can be filtered like it appears the other selectors are being filtered). For example, with the following rmap:
https://jwst-crds.stsci.edu/browse/jwst_miri_mask_0022.rmap
the selector:
('MIRIFULONG', 'GENERIC', 'FAST|FASTR1', 'BETWEEN 0 50') : UseAfter({
'2022-04-01 00:00:00' : 'jwst_miri_mask_0053.fits',
'2022-07-12 17:08:30' : 'jwst_miri_mask_0057.fits',
'2023-02-01 16:54:24' : 'jwst_miri_mask_0052.fits',
'2023-09-28 16:04:12' : 'jwst_miri_mask_0054.fits',
}),
Contains a tuple selector (I'm not sure I'm using terms here that match the crds conventions).
('MIRIFULONG', 'GENERIC', 'FAST|FASTR1', 'BETWEEN 0 50')
Note the last two items FAST|FASTR1
and BETWEEN 0 50
corresponding to READPATT
and NGROUPS
. Looking at the 4 selected files, all contain FASTR1
for READPATT
and BETWEEN 0 50
for NGROUPS
. This led me to a few questions:
- Why is the
READPATT
selectorFAST|FASTR1
if all files areFASTR1
? - Does crds generate the selector from the file(s) or is this separately defined?
It's possible this is related to the 'tpn' files which does have a constraint for READPATT
but not for NGROUPS
:
https://jwst-crds.stsci.edu/browse/jwst_miri_mask_0057.fits
or with the crds submission process where possibly READPATT
is filtered and NGROUPS
is not.