AxFoundation/strax

Double dependencies freez mailbox system

Closed this issue · 2 comments

Problem:
In some studies we noticed a peak reconstruction bias due to our peak merging procedure. Currently, if we merge two peaklets with a lone hit in between the information of the lone hit is not added to the peak. This leads to an area bias of a few percent in the reconstructed peaks. In order to solve this issue I made #506 and XENONnT/straxen#623. These changes require a double dependency of merged_s2s on peaklets (see also attached graph). In addition peaklets and lone_hits are chunked differently.

This double dependency is currently not properly supported and let our mailbox system end up in a dead lock when processing up to merged_s2s without pre-processing peaklets. If peaklets are made before calling st.make(.... merged_s2s) things work as expected.

example

Let me also add the debugging logging info:

DEBUG:strax:Final plugins: ('merged_s2s',)
DEBUG:ThreadedMailboxProcessor:Processor components are: ProcessorComponents(plugins={'merged_s2s': MergedS2s, 'peaklets': Peaklets, 'peaklet_classification': PeakletClassification, 'lone_hits': Peaklets}, loaders={'records': functools.partial(<bound method StorageFrontend.loader of strax.storage.files.DataDirectory, path: /dali/lgrandi/wenz/strax_data/strax_tests>, 025258-records-wowj65ge6b, time_range=None, chunk_number=None, fuzzy_for=(), fuzzy_for_options=(), allow_incomplete=False)}, savers={'peaklets': [<strax.storage.files.FileSaver object at 0x7fc4242eefa0>], 'lone_hits': [<strax.storage.files.FileSaver object at 0x7fc4242feeb0>], 'peaklet_classification': [<strax.storage.files.FileSaver object at 0x7fc45c6be340>], 'merged_s2s': [<strax.storage.files.FileSaver object at 0x7fc4242eebb0>]}, targets=('merged_s2s',))
DEBUG:ThreadedMailboxProcessor:to_flow_freely set()to_discard set()produced {'records', 'merged_s2s', 'peaklets', 'lone_hits', 'peaklet_classification'}required {'records', 'merged_s2s', 'peaklets', 'lone_hits', 'peaklet_classification'}saved {'merged_s2s', 'peaklets', 'lone_hits', 'peaklet_classification'}
DEBUG:records_mailbox:Initialized
DEBUG:merged_s2s_mailbox:Initialized
DEBUG:peaklets_mailbox:Initialized
DEBUG:peaklets_mailbox:Added subscriber 0
DEBUG:peaklet_classification_mailbox:Initialized
DEBUG:peaklet_classification_mailbox:Added subscriber 0
DEBUG:lone_hits_mailbox:Initialized
DEBUG:lone_hits_mailbox:Added subscriber 0
DEBUG:ThreadedMailboxProcessor:multi_output_seen: [Peaklets]
DEBUG:Peaklets_divide_outputs_mailbox:Initialized
DEBUG:records_mailbox:Added subscriber 0
DEBUG:Peaklets_divide_outputs_mailbox:Added subscriber 0
DEBUG:peaklets_mailbox:Added subscriber 1
DEBUG:ThreadedMailboxProcessor:multi_output_seen: [Peaklets, Peaklets]
DEBUG:records_mailbox:Added subscriber 1
DEBUG:Peaklets_divide_outputs_mailbox:Added subscriber 1
DEBUG:peaklets_mailbox:Added subscriber 2
DEBUG:lone_hits_mailbox:Added subscriber 1
DEBUG:peaklet_classification_mailbox:Added subscriber 1
DEBUG:merged_s2s_mailbox:Added subscriber 0
DEBUG:merged_s2s_mailbox:Added subscriber 1
DEBUG:ThreadedMailboxProcessor:Starting threads
DEBUG:ThreadedMailboxProcessor:start <Mailbox: records_mailbox>
DEBUG:records_mailbox:Waiting to fetch 0, [None, None], [True, True]
DEBUG:ThreadedMailboxProcessor:start <Mailbox: merged_s2s_mailbox>
DEBUG:merged_s2s_mailbox:Waiting to fetch 0, [None, None], [False, True]
DEBUG:merged_s2s_mailbox:Start reading
DEBUG:merged_s2s_mailbox:Checking/waiting for 0
DEBUG:ThreadedMailboxProcessor:start <Mailbox: peaklets_mailbox>
DEBUG:peaklets_mailbox:Start reading
DEBUG:ThreadedMailboxProcessor:start <Mailbox: peaklet_classification_mailbox>
DEBUG:peaklets_mailbox:Checking/waiting for 0
DEBUG:peaklet_classification_mailbox:Waiting to fetch 0, [None, None], [True, False]
DEBUG:peaklet_classification_mailbox:Start reading
DEBUG:ThreadedMailboxProcessor:start <Mailbox: lone_hits_mailbox>
DEBUG:peaklet_classification_mailbox:Checking/waiting for 0
DEBUG:lone_hits_mailbox:Start reading
DEBUG:ThreadedMailboxProcessor:start <Mailbox: Peaklets_divide_outputs_mailbox>
DEBUG:lone_hits_mailbox:Checking/waiting for 0
DEBUG:Peaklets_divide_outputs_mailbox:Waiting to fetch 0, [None, None], [True, True]
DEBUG:peaklets_mailbox:Waiting to fetch 0, [None, None, 0], [True, True, False]
DEBUG:Peaklets_divide_outputs_mailbox:Waiting to fetch 0, [None, None], [True, True]
DEBUG:peaklets_mailbox:Waiting to fetch 0, [None, None, 0], [True, True, False]
DEBUG:ThreadedMailboxProcessor:Yielding merged_s2s
DEBUG:merged_s2s_mailbox:Start reading
DEBUG:merged_s2s_mailbox:Checking/waiting for 0
DEBUG:peaklets_mailbox:Start reading
DEBUG:peaklets_mailbox:Checking/waiting for 0
DEBUG:lone_hits_mailbox:Waiting to fetch 0, [None, 0], [True, False]
DEBUG:lone_hits_mailbox:Waiting to fetch 0, [None, 0], [True, False]

Should be solved with #512