swiss-seismological-service/scdetect

minimumArrivals

mmesim opened this issue · 10 comments

mmesim commented

@damb Well, I shared with you the files. Here I'll report the problem. I run the shared example and set minimumArrivals: 10. However, when I run scevent with the results.scml I received the following message multiple times

rejecting automatic origin Origin/20220318161026.734837.31225 (phaseCount: 3 < 10)

or

rejecting automatic origin Origin/20220318161026.734837.31225 (phaseCount: 6 < 10)

That probably means that there are not 10 picks available, which seems weird.

I still do not if it scdetect-cc or scevent problem.

I didn't include the results.scml to check whether you can fully reproduce the error.

damb commented

@mmesim, thanks for reporting.

That probably means that there are not 10 picks available, which seems weird.

Without having checked the reproducer's configuration yet, I guess that this might be an issue when trying to avoid to associate a single phase to be associated multiple times with a sensor location. This is required in certain configuration scenarios, e.g. given the following detector configuration:

  {
    "detectorId": "detector-01",
    "createArrivals": true,
    "originId": "originId",
    "templatePhase": "Sg",
    "templateWaveformStart": -0.25,
    "templateWaveformEnd": 2,
    "streams": [
      {
        "templateId": "template-01",
        "waveformId": "CH.GRIMS..HHN"
      },
      {
        "templateId": "template-02",
        "waveformId": "CH.GRIMS..HHE"
      }
    ]
  }

I.e. in the example above, in order to avoid to associate an Arrival with phase Sg twice (regarding the sensor location CH.GRIMS..) scdetect-cc drops one of the arrivals including the respective pick. This is not ideal, however, AFAIU, in the SeisComP datamodel the relationship Pick <-> Origin can be established only by means of an Arrival.

Do you have a better idea how to circumvent this issue?

mmesim commented

What I do not understand is why I didn't face this problem with the other testing datasets.

If what you mention above is the case, then it should have happened before. Still thinking about it...

damb commented

What I do not understand is why I didn't face this problem with the other testing datasets.

I'll have a look into the reproducer and come back.

  • 👍1
damb commented

If what you mention above is the case, then it should have happened before. Still thinking about it...

  • You mean, while using the "minimumArrivals" configration parameter, as well?
  • Did you check scevent logs previously, too?
mmesim commented

You mean, while using the "minimumArrivals" configration parameter, as well?

Yes.

Did you check scevent logs previously, too?

Yes

damb commented

Ok. So when trying to initialize detector-01 with

$ mkdir data.sds && \
scart -I data.mseed data.sds && \
scdetect-cc --templates-prepare --debug --offline --playback --ep=detections.scml --templates-json=$(realpath templates.json) --inventory-db="file://$(realpath inventory.scml)" --record-url="sdsarchive://$(realpath data.sds)" --event-db="file://$(realpath catalog.scml)" --config-db="file://$(realpath config.scml)"   --amplitudes-force=0

I obtain the following logs:

Logs 
14:36:45 [debug] Adding plugin path: .
14:36:45 [debug] Adding plugin path: /home/damb/.seiscomp/plugins
14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/lib/plugins
14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/lib
14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/share/plugins
14:36:45 [debug] Adding plugin path: /home/damb/seiscomp/share/plugins/scdetect-cc
14:36:45 [debug] Trying to open plugin at /home/damb/seiscomp/share/plugins/dbpostgresql.so
14:36:45 [info] Plugin dbpostgresql registered
14:36:45 [info] 
Plugins:
--------
 [1]
  description: PostgreSQL database driver
       author: GFZ Potsdam <seiscomp-devel@gfz-potsdam.de>
      version: 0.11.0
          API: 15.0.0

14:36:45 [info] Loading complete inventory from file:///home/damb/tmp/scdetect/iss-101/stations.scml
14:36:45 [info] Finished loading complete inventory
14:36:45 [debug] trying to open stream file:///home/damb/tmp/scdetect/iss-101/data.mseed
14:36:45 [info] Playback mode enabled
14:36:45 [info] Loading events from file:///home/damb/tmp/scdetect/iss-101/events.scml
14:36:45 [debug] Found database version v0.12
14:36:46 [debug] Found database version v0.12
14:36:46 [info] Finished loading events
14:36:46 [info] Loading template configuration from /home/damb/tmp/scdetect/iss-101/templates.json
14:36:46 [debug] Creating detector processor (id=detector-01) ... 
14:36:46 [debug] [8D.ELM1..EHZ (8D.ELM1..EHZ)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:39.014947Z, phase=Pg, stream=8D.ELM1..EHZ
14:36:46 [debug] [8D.ELM1..EHZ (8D.ELM1..EHZ)] loaded stream from inventory for epoch: start=2020-10-25T19:38:38.764948Z, end=2020-10-25T19:38:40.014947Z
14:36:46 [debug] [8D.ELM1..EHZ (8D.ELM1..EHZ)] creating template waveform processor with id: detector-01::template-10
14:36:46 [debug] [detector-01::template-10] [8D.ELM1..EHZ (8D.ELM1..EHZ)] filters configured: filter="BW_BP(2,1.5,15)"
14:36:46 [warning] [CH.STLM2..HGN (CH.STLM2..HGN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [CH.STLM2..HGZ (CH.STLM2..HGZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization.
14:36:46 [debug] [CH.PANIX..HHN (CH.PANIX..HHN)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.111906Z, phase=Sg, stream=CH.PANIX..HHE
14:36:46 [debug] [CH.PANIX..HHN (CH.PANIX..HHN)] loaded stream from inventory for epoch: start=2020-10-25T19:38:39.861907Z, end=2020-10-25T19:38:42.111906Z
14:36:46 [debug] [CH.PANIX..HHN (CH.PANIX..HHN)] creating template waveform processor with id: detector-01::template-06
14:36:46 [debug] [detector-01::template-06] [CH.PANIX..HHN (CH.PANIX..HHN)] filters configured: filter="BW_BP(2,1.5,15)"
14:36:46 [debug] [CH.PANIX..HHE (CH.PANIX..HHE)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.111906Z, phase=Sg, stream=CH.PANIX..HHE
14:36:46 [debug] [CH.PANIX..HHE (CH.PANIX..HHE)] loaded stream from inventory for epoch: start=2020-10-25T19:38:39.861907Z, end=2020-10-25T19:38:42.111906Z
14:36:46 [debug] [CH.PANIX..HHE (CH.PANIX..HHE)] creating template waveform processor with id: detector-01::template-05
14:36:46 [debug] [detector-01::template-05] [CH.PANIX..HHE (CH.PANIX..HHE)] filters configured: filter="BW_BP(2,1.5,15)"
14:36:46 [debug] [CH.PANIX..HHZ (CH.PANIX..HHZ)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:38.84245Z, phase=Pg, stream=CH.PANIX..HHZ
14:36:46 [debug] [CH.PANIX..HHZ (CH.PANIX..HHZ)] loaded stream from inventory for epoch: start=2020-10-25T19:38:38.592451Z, end=2020-10-25T19:38:39.84245Z
14:36:46 [debug] [CH.PANIX..HHZ (CH.PANIX..HHZ)] creating template waveform processor with id: detector-01::template-04
14:36:46 [debug] [detector-01::template-04] [CH.PANIX..HHZ (CH.PANIX..HHZ)] filters configured: filter="BW_BP(2,1.5,15)"
14:36:46 [debug] [8D.ELM1..EHN (8D.ELM1..EHN)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.331748Z, phase=Sg, stream=8D.ELM1..EHE
14:36:46 [debug] [8D.ELM1..EHN (8D.ELM1..EHN)] loaded stream from inventory for epoch: start=2020-10-25T19:38:40.081749Z, end=2020-10-25T19:38:42.331748Z
14:36:46 [debug] [8D.ELM1..EHN (8D.ELM1..EHN)] creating template waveform processor with id: detector-01::template-12
14:36:46 [debug] [detector-01::template-12] [8D.ELM1..EHN (8D.ELM1..EHN)] filters configured: filter="BW_BP(2,1.5,15)"
14:36:46 [debug] [8D.ELM1..EHE (8D.ELM1..EHE)] using arrival pick: origin=smi:ch.ethz.sed/sc3a/origin/NLL.20201111122232.712663.968378, time=2020-10-25T19:38:40.331748Z, phase=Sg, stream=8D.ELM1..EHE
14:36:46 [debug] [8D.ELM1..EHE (8D.ELM1..EHE)] loaded stream from inventory for epoch: start=2020-10-25T19:38:40.081749Z, end=2020-10-25T19:38:42.331748Z
14:36:46 [debug] [8D.ELM1..EHE (8D.ELM1..EHE)] creating template waveform processor with id: detector-01::template-11
14:36:46 [debug] [detector-01::template-11] [8D.ELM1..EHE (8D.ELM1..EHE)] filters configured: filter="BW_BP(2,1.5,15)"
14:36:46 [warning] [8D.EML0..EHN (8D.EML0..EHN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHE (8D.EML0..EHE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [CH.STLM2..HGE (CH.STLM2..HGE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHZ (8D.EML0..EHZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization.
14:36:46 [debug] Application initialized
14:36:46 [debug] Subscribing to streams required for processing
14:36:46 [debug] Subscribing to stream: 8D.ELM1..EHE
14:36:46 [debug] Subscribing to stream: 8D.ELM1..EHN
14:36:46 [debug] Subscribing to stream: 8D.ELM1..EHZ
14:36:46 [debug] Subscribing to stream: CH.PANIX..HHE
14:36:46 [debug] Subscribing to stream: CH.PANIX..HHN
14:36:46 [debug] Subscribing to stream: CH.PANIX..HHZ
14:36:46 [info] Starting record acquisition
14:36:46 [debug] [detector-01::template-11] Initialize stream: sampling_frequency=200.000000
14:36:46 [debug] [detector-01::template-12] Initialize stream: sampling_frequency=200.000000
14:36:46 [debug] [detector-01::template-10] Initialize stream: sampling_frequency=200.000000
14:36:46 [debug] [detector-01::template-05] Initialize stream: sampling_frequency=200.000000
14:36:46 [debug] [detector-01::template-04] Initialize stream: sampling_frequency=200.000000
14:36:46 [debug] [detector-01::template-06] Initialize stream: sampling_frequency=200.000000
14:36:48 [debug] [detector-01] Detector result (triggering) score=1.000000, associated_results=6
^C14:36:50 [debug] [detector-01] Terminating ...
14:36:50 [debug] [detector-01] Start processing detection (time=2020-10-25T19:38:37.068104Z, associated_results=6) ...
14:36:50 [debug] Found 1 origins.
14:36:50 [debug] Leaving ::done
14:36:50 [info] Waiting for record thread
14:36:50 [debug] Unload plugin 'PostgreSQL database driver'

The warnings

14:36:46 [warning] [CH.STLM2..HGN (CH.STLM2..HGN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [CH.STLM2..HGZ (CH.STLM2..HGZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHN (8D.EML0..EHN)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHE (8D.EML0..EHE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [CH.STLM2..HGE (CH.STLM2..HGE)] sensor location not found in inventory for time: 2020-10-25T19:38:48.784752Z. Skipping template waveform processor initialization.
14:36:46 [warning] [8D.EML0..EHZ (8D.EML0..EHZ)] sensor location not found in inventory for time: 2020-10-25T19:38:37.967674Z. Skipping template waveform processor initialization.

indicate that the inventory metadata was not available for the corresponding streams and time window. The detector is initialized, anyway, however, just with 6 streams. Due to the behaviour described previously (i.e. in https://github.com/damb/scdetect/issues/101#issuecomment-1073745094), only 4 picks will be associated in case of a detection.

Does this help?

mmesim commented

I just realized that there was a typo. Instead of writing ELM0 I wrote EML0. 😮‍💨

Also, I do not know why STLM2.HG? is not in the inventory. Perhaps I did not include acc? I don't remember.

To conclude, in the case that scdetect-cc will not be able to identify 10 phases, what will it do and how is this trackable/manageable in real time application?

damb commented

To conclude, in the case that scdetect-cc will not be able to identify 10 phases, what will it do and how is this trackable/manageable in real time application?

Warnings are already issued. Though, in case the "minimumArrivals" is configured and the value is greater than the number of streams available scdetect-cc may not initialize the detector at all. What do you suggest?

EDIT: Currently, scdetect-cc does initialize the detector, but it resets the "minimumArrivals" to the number of streams available.

mmesim commented

Need to think about it.

  • 👀1
damb commented

Ok, this should be solved. Thanks for the discussion, @mmesim.

As a small enhancement, we're going to implement #102.