datumRanges containing fill data
Opened this issue · 4 comments
Up until now it's been allowed for DatumRanges to contain fill values. While this might cause many codes to break I think we need to consider disallowing this. These are typically something you need to deal with anyway, and the maxim fail early so that bugs don't reveal themselves far away from their source says this should be done.
https://github.com/autoplot/dev/blob/master/bugs/2023/20230210/demoDatumRangeFill.jy
I forgot to push this change initially and I see this breaks a lot of things. union(d1,d2) where d2 is fill now returns the union(d1,d1) and likewise for the other.
There's another problem, where -1e31 is still assumed to be fill with Datums. Autoplot test037 shows where a CDF can't be loaded because the dataset contains -1e31, which is not properly marked as fill in the CDF, and then when this becomes a Datum it is then treated as fill. This should be cleaned up as well.
I'm starting to see again how difficult this is, and it will require touching hundreds of codes to implement completely. Another way to deal with this is to allow fill to represent open-ended, so that [1,fill] would be a number ge 1. This would be useful too, and provides a clear definition.
This is a guess at the most functional logic:
| in | out | comments |
|---|---|---|
| union(fill,1) | dr([1,1]) | |
| union(dr(fill,1),0) | dr(fill,1) | |
| union(dr(fill,1),2) | dr(fill,2) | |
| union(dr(fill,0),dr(0,fill) | dr(fill,fill) | |
| intersection(dr(fill,1),dr(0,fill) | dr(0,1) | |
| dr(fill,1).next() | dr(fill,1) | any fill will return the same datumrange |