Bug to explain 12 seasons assignment to 10 year data
Closed this issue · 7 comments
When I run this stacker on OpSim 2.168 data, I get a total of 12 seasons assigned to the visits for 10 years, with a considerable number of observations in the 12th season. Looking closely at the code, it appears that there’s a 6-month subtraction issue: since using sunRA equinox already subtracts 6 months off of objectRA equinox, the 6 month subtraction in line 53 is repetitive. However, removing the subtraction in line 53 doesn’t fix the issue.
To check whether my understanding is correct, I have used objectRA to determine the seasons, which gives me 11 seasons (I am wrapping the first and 11th one to get a total of 10 seasons). I am guessing there’s more of a subtle bug in the sunRA code that I have not been able to discover so far.
Won't some fields - the ones that are up during the time of the data
release, the turn of the year - be observed for 11 seasons? eg, I'm
thinking of a field with a 4-month continuous observing season that starts
being observed in July, and gets 2 months data before the end of the year
that is released as DR1. It then gets two more months in September and
October, but these will be released in DR2 (along with the following July
and August's data). In each data release, then, there are 4 months of data
for this field: DR1 contains September and October, and then this following
July and August. This first year should be counted as 2 seasons though,
separated (and defined) by the usual long season gap. In year 10, there's
another short 2-month season in July and August, and then LSST is shut
down. I make that 11 seasons total: two short seasons and 9 long ones. Only
some ( about a third?) of the fields should show this behavior.
On Thu, Jul 9, 2015 at 10:02 PM, humnaawan notifications@github.com wrote:
When I run this stacker on OpSim 2.168 data, I get a total of 12 seasons
assigned to the visits for 10 years, with a considerable number of
observations in the 12th season. Looking closely at the code, it appears
that there’s a 6-month subtraction issue: since using sunRA equinox already
subtracts 6 months off of objectRA equinox, the 6 month subtraction in line
53 is repetitive. However, removing the subtraction in line 53 doesn’t fix
the issue.To check whether my understanding is correct, I have used objectRA to
determine the seasons, which gives me 11 seasons (I am wrapping the first
and 11th one to get a total of 10 seasons). I am guessing there’s more of a
subtle bug in the sunRA code that I have not been able to discover so far.—
Reply to this email directly or view it on GitHub
#14.
Hi Humna - Phil is the author of the season stacker, so it's great that the two of you are discussing this.
@drphilmarshall .. in your example, you still only get 11 seasons. Does this mean that the 12 seasons result really is a bug and should be updated? (and then some fields will have 10 and some will have 11 seasons?)
I think so, yes: we expect most fields to have 10 seasons and some to hve
11. I thought this is what I was seeing... But I can definitely believe I
got some dates and/or angles wrong! :-)
On Monday, July 13, 2015, Lynne Jones notifications@github.com wrote:
Hi Humna - Phil is the author of the season stacker, so it's great that
the two of you are discussing this.
@drphilmarshall https://github.com/drphilmarshall .. in your example,
you still only get 11 seasons. Does this mean that the 12 seasons result
really is a bug and should be updated? (and then some fields will have 10
and some will have 11 seasons?)—
Reply to this email directly or view it on GitHub
#14 (comment)
.
Just to illustrate what I am looking at, here’s the season assignment for Opsim 2.168 main survey data with your seasonStacker:
Total visits for all fields: 363185
season 0.0
numEntries 1659 ; 0.456791993061 % of total
Corresponding years: [ 0.]
season 1.0
numEntries 14414 ; 3.96877624351 % of total
Corresponding years: [ 0. 1.]
season 2.0
numEntries 18835 ; 5.18606219971 % of total
Corresponding years: [ 1. 2.]
season 3.0
numEntries 34474 ; 9.49213210898 % of total
Corresponding years: [ 2. 3.]
season 4.0
numEntries 40433 ; 11.1328937043 % of total
Corresponding years: [ 2. 3. 4.]
season 5.0
numEntries 45055 ; 12.4055233559 % of total
Corresponding years: [ 4. 5.]
season 6.0
numEntries 55098 ; 15.1707807316 % of total
Corresponding years: [ 5. 6.]
season 7.0
numEntries 46778 ; 12.8799372221 % of total
Corresponding years: [ 6. 7.]
season 8.0
numEntries 44655 ; 12.2953866487 % of total
Corresponding years: [ 7. 8.]
season 9.0
numEntries 35113 ; 9.66807549871 % of total
Corresponding years: [ 8. 9.]
season 10.0
numEntries 26650 ; 7.33785811639 % of total
Corresponding years: [ 8. 9.]
season 11.0
numEntries 21 ; 0.00578217712736 % of total
Corresponding years: [ 9.]
—
And here’s the output with enigma_1189 database (WFD only):
Total visits for all fields: 463866
season 0.0
numEntries 922 ; 0.19876429831 % of total
Corresponding years: [ 0.]
season 1.0
numEntries 23571 ; 5.08142437687 % of total
Corresponding years: [ 0. 1.]
season 2.0
numEntries 38351 ; 8.26768937581 % of total
Corresponding years: [ 0. 1. 2.]
season 3.0
numEntries 45258 ; 9.75669697714 % of total
Corresponding years: [ 1. 2. 3.]
season 4.0
numEntries 48965 ; 10.5558501809 % of total
Corresponding years: [ 3. 4.]
season 5.0
numEntries 54735 ; 11.7997438916 % of total
Corresponding years: [ 3. 4. 5.]
season 6.0
numEntries 55205 ; 11.9010662562 % of total
Corresponding years: [ 4. 5. 6.]
season 7.0
numEntries 50966 ; 10.9872247589 % of total
Corresponding years: [ 5. 6. 7.]
season 8.0
numEntries 54809 ; 11.8156967745 % of total
Corresponding years: [ 6. 7. 8.]
season 9.0
numEntries 50970 ; 10.9880870769 % of total
Corresponding years: [ 7. 8. 9.]
season 10.0
numEntries 39989 ; 8.62080859559 % of total
Corresponding years: [ 8. 9.]
season 11.0
numEntries 125 ; 0.0269474374065 % of total
Corresponding years: [ 9.]
—
As you can see, both give a total of 12 seasons, though only a few entries are associated with the last one.
—
Now after fixing the bug, I am getting this for Opsim 2.168 main survey:
Total visits for all fields: 363185
season 0.0
numEntries 7102 ; 1.95547723612 % of total
Corresponding years: [ 0.]
season 1.0
numEntries 17264 ; 4.75350028223 % of total
Corresponding years: [ 0. 1.]
season 2.0
numEntries 22862 ; 6.29486349932 % of total
Corresponding years: [ 1. 2.]
season 3.0
numEntries 39920 ; 10.9916433773 % of total
Corresponding years: [ 2. 3.]
season 4.0
numEntries 39829 ; 10.9665872765 % of total
Corresponding years: [ 3. 4.]
season 5.0
numEntries 52825 ; 14.544928893 % of total
Corresponding years: [ 4. 5.]
season 6.0
numEntries 50581 ; 13.9270619657 % of total
Corresponding years: [ 5. 6.]
season 7.0
numEntries 47331 ; 13.0322012198 % of total
Corresponding years: [ 6. 7.]
season 8.0
numEntries 45030 ; 12.3986398117 % of total
Corresponding years: [ 7. 8.]
season 9.0
numEntries 31853 ; 8.77046133513 % of total
Corresponding years: [ 8. 9.]
season 10.0
numEntries 8588 ; 2.36463510332 % of total
Corresponding years: [ 9.]
and this for enigma:
Total visits for all fields: 463866
season 0.0
numEntries 8021 ; 1.7291631635 % of total
Corresponding years: [ 0.]
season 1.0
numEntries 35716 ; 7.69963739528 % of total
Corresponding years: [ 0. 1.]
season 2.0
numEntries 37596 ; 8.10492685388 % of total
Corresponding years: [ 1. 2.]
season 3.0
numEntries 49007 ; 10.5649045198 % of total
Corresponding years: [ 2. 3. 4.]
season 4.0
numEntries 49955 ; 10.7692738851 % of total
Corresponding years: [ 3. 4.]
season 5.0
numEntries 57685 ; 12.4357034143 % of total
Corresponding years: [ 4. 5. 6.]
season 6.0
numEntries 52092 ; 11.229967275 % of total
Corresponding years: [ 5. 6.]
season 7.0
numEntries 53997 ; 11.6406462211 % of total
Corresponding years: [ 6. 7.]
season 8.0
numEntries 52876 ; 11.3989816024 % of total
Corresponding years: [ 7. 8.]
season 9.0
numEntries 49710 ; 10.7164569078 % of total
Corresponding years: [ 8. 9.]
season 10.0
numEntries 17211 ; 3.71033876163 % of total
Corresponding years: [ 9.]
—
For our dithering purposes, we are wrapping the 0th and 10th seasons, giving us a total of 10 seasons. Since I am doing this wrapping in my dithering stackers and not in my version of the seasonStacker, you’ll still get 11 seasons for your lensing studies.
Would it be okay if I push my version of the SeasonStacker? I currently have it named as SeasonStacker_v2, and the header details the difference in strategy. Please let me know if that’d be okay.
Looks like your bug fix does indeed make things cleaner. I'm happy for you
to push it, Humna. Or if you're working in a separate branch, submit a pull
request and I'll merge it.
Thanks for posting the examples, they were interesting. I wonder why some
seasons have data from three years in them? That is not supposed to
happen...
On Wed, Jul 15, 2015 at 2:31 AM, humnaawan notifications@github.com wrote:
Just to illustrate what I am looking at, here’s the season assignment for
Opsim 2.168 main survey data with your seasonStacker:Total visits for all fields: 363185
season 0.0
numEntries 1659 ; 0.456791993061 % of total
Corresponding years: [ 0.]season 1.0
numEntries 14414 ; 3.96877624351 % of total
Corresponding years: [ 0. 1.]season 2.0
numEntries 18835 ; 5.18606219971 % of total
Corresponding years: [ 1. 2.]season 3.0
numEntries 34474 ; 9.49213210898 % of total
Corresponding years: [ 2. 3.]season 4.0
numEntries 40433 ; 11.1328937043 % of total
Corresponding years: [ 2. 3. 4.]season 5.0
numEntries 45055 ; 12.4055233559 % of total
Corresponding years: [ 4. 5.]season 6.0
numEntries 55098 ; 15.1707807316 % of total
Corresponding years: [ 5. 6.]season 7.0
numEntries 46778 ; 12.8799372221 % of total
Corresponding years: [ 6. 7.]season 8.0
numEntries 44655 ; 12.2953866487 % of total
Corresponding years: [ 7. 8.]season 9.0
numEntries 35113 ; 9.66807549871 % of total
Corresponding years: [ 8. 9.]season 10.0
numEntries 26650 ; 7.33785811639 % of total
Corresponding years: [ 8. 9.]season 11.0
numEntries 21 ; 0.00578217712736 % of total
Corresponding years: [ 9.]—
And here’s the output with enigma_1189 database (WFD only):Total visits for all fields: 463866
season 0.0
numEntries 922 ; 0.19876429831 % of total
Corresponding years: [ 0.]season 1.0
numEntries 23571 ; 5.08142437687 % of total
Corresponding years: [ 0. 1.]season 2.0
numEntries 38351 ; 8.26768937581 % of total
Corresponding years: [ 0. 1. 2.]season 3.0
numEntries 45258 ; 9.75669697714 % of total
Corresponding years: [ 1. 2. 3.]season 4.0
numEntries 48965 ; 10.5558501809 % of total
Corresponding years: [ 3. 4.]season 5.0
numEntries 54735 ; 11.7997438916 % of total
Corresponding years: [ 3. 4. 5.]season 6.0
numEntries 55205 ; 11.9010662562 % of total
Corresponding years: [ 4. 5. 6.]season 7.0
numEntries 50966 ; 10.9872247589 % of total
Corresponding years: [ 5. 6. 7.]season 8.0
numEntries 54809 ; 11.8156967745 % of total
Corresponding years: [ 6. 7. 8.]season 9.0
numEntries 50970 ; 10.9880870769 % of total
Corresponding years: [ 7. 8. 9.]season 10.0
numEntries 39989 ; 8.62080859559 % of total
Corresponding years: [ 8. 9.]season 11.0
numEntries 125 ; 0.0269474374065 % of total
Corresponding years: [ 9.]—
As you can see, both give a total of 12 seasons, though only a few entries
are associated with the last one.
—
Now after fixing the bug, I am getting this for Opsim 2.168 main survey:Total visits for all fields: 363185
season 0.0
numEntries 7102 ; 1.95547723612 % of total
Corresponding years: [ 0.]season 1.0
numEntries 17264 ; 4.75350028223 % of total
Corresponding years: [ 0. 1.]season 2.0
numEntries 22862 ; 6.29486349932 % of total
Corresponding years: [ 1. 2.]season 3.0
numEntries 39920 ; 10.9916433773 % of total
Corresponding years: [ 2. 3.]season 4.0
numEntries 39829 ; 10.9665872765 % of total
Corresponding years: [ 3. 4.]season 5.0
numEntries 52825 ; 14.544928893 % of total
Corresponding years: [ 4. 5.]season 6.0
numEntries 50581 ; 13.9270619657 % of total
Corresponding years: [ 5. 6.]season 7.0
numEntries 47331 ; 13.0322012198 % of total
Corresponding years: [ 6. 7.]season 8.0
numEntries 45030 ; 12.3986398117 % of total
Corresponding years: [ 7. 8.]season 9.0
numEntries 31853 ; 8.77046133513 % of total
Corresponding years: [ 8. 9.]season 10.0
numEntries 8588 ; 2.36463510332 % of total
Corresponding years: [ 9.]and this for enigma:
Total visits for all fields: 463866
season 0.0
numEntries 8021 ; 1.7291631635 % of total
Corresponding years: [ 0.]season 1.0
numEntries 35716 ; 7.69963739528 % of total
Corresponding years: [ 0. 1.]season 2.0
numEntries 37596 ; 8.10492685388 % of total
Corresponding years: [ 1. 2.]season 3.0
numEntries 49007 ; 10.5649045198 % of total
Corresponding years: [ 2. 3. 4.]season 4.0
numEntries 49955 ; 10.7692738851 % of total
Corresponding years: [ 3. 4.]season 5.0
numEntries 57685 ; 12.4357034143 % of total
Corresponding years: [ 4. 5. 6.]season 6.0
numEntries 52092 ; 11.229967275 % of total
Corresponding years: [ 5. 6.]season 7.0
numEntries 53997 ; 11.6406462211 % of total
Corresponding years: [ 6. 7.]season 8.0
numEntries 52876 ; 11.3989816024 % of total
Corresponding years: [ 7. 8.]season 9.0
numEntries 49710 ; 10.7164569078 % of total
Corresponding years: [ 8. 9.]season 10.0
numEntries 17211 ; 3.71033876163 % of total
Corresponding years: [ 9.]—
For our dithering purposes, we are wrapping the 0th and 10th seasons,
giving us a total of 10 seasons. Since I am doing this wrapping in my
dithering stackers and not in my version of the seasonStacker, you’ll still
get 11 seasons for your lensing studies.Would it be okay if I push my version of the SeasonStacker? I currently
have it named as SeasonStacker_v2, and the header details the difference in
strategy. Please let me know if that’d be okay.—
Reply to this email directly or view it on GitHub
#14 (comment)
.
Phil, the three season occurrence does not seem that common. I ran the code to output number of entries in each year in each season. As you can see, the number of entries in the third year is quite minimal:
Running: ['PentagonDitherPerSeason']
Seasons to wrap [ 10.] with total entries: 17211
Total visits for all fields: 463866
season 0.0
numEntries 25232 ; 5.43950192512 % of total
Corresponding years: [ 0. 9.]
Entries in year 0.0 : 8021
Entries in year 9.0 : 17211
season 1.0
numEntries 35716 ; 7.69963739528 % of total
Corresponding years: [ 0. 1.]
Entries in year 0.0 : 9139
Entries in year 1.0 : 26577
season 2.0
numEntries 37596 ; 8.10492685388 % of total
Corresponding years: [ 1. 2.]
Entries in year 1.0 : 12446
Entries in year 2.0 : 25150
season 3.0
numEntries 49007 ; 10.5649045198 % of total
Corresponding years: [ 2. 3. 4.]
Entries in year 2.0 : 14796
Entries in year 3.0 : 34210
Entries in year 4.0 : 1
season 4.0
numEntries 49955 ; 10.7692738851 % of total
Corresponding years: [ 3. 4.]
Entries in year 3.0 : 11998
Entries in year 4.0 : 37957
season 5.0
numEntries 57685 ; 12.4357034143 % of total
Corresponding years: [ 4. 5. 6.]
Entries in year 4.0 : 19053
Entries in year 5.0 : 38630
Entries in year 6.0 : 2
season 6.0
numEntries 52092 ; 11.229967275 % of total
Corresponding years: [ 5. 6.]
Entries in year 5.0 : 17503
Entries in year 6.0 : 34589
season 7.0
numEntries 53997 ; 11.6406462211 % of total
Corresponding years: [ 6. 7.]
Entries in year 6.0 : 16119
Entries in year 7.0 : 37878
season 8.0
numEntries 52876 ; 11.3989816024 % of total
Corresponding years: [ 7. 8.]
Entries in year 7.0 : 16372
Entries in year 8.0 : 36504
season 9.0
numEntries 49710 ; 10.7164569078 % of total
Corresponding years: [ 8. 9.]
Entries in year 8.0 : 15241
Entries in year 9.0 : 34469
Note that I have a total of only 10 seasons (0-9), as I am wrapping the 0th and 10th seasons for dithering purposes.
I think the problem might be arising from a few fields not being observed every season. It is something we've noticed as we were implementing dithers for all fields every season vs. for each field every season it is observed.
Also, I have pushed the stacker. Hope it is available to view.
The stacker (version after Humna's fix) is now available in the core sims.maf.stackers.