ipfx should handle sweeps with missing stimulus amplitudes

[wbwakeman]

NWB2 files are being generated by the ephys rigs at AIBS now and being sent to LIMS. Most, or all, are failing because one of the sweeps has a null stimulus amplitude. Example shown below

Rusty says:

“this is something that does occur quite regularly as a byproduct of the real time QC functions that we are running during acquisition. If a sweep is started and the cell’s pre-stimulus baseline voltage is beyond +/- 1mV of the predetermined resting baseline, then the sweep will be terminated early, in many cases before the stimulus epoch is initiated, and so those sweeps will not have a stimulus amplitude associated with them.”

/allen/programs/celltypes/production/mousecelltypes/prod3000/Ephys_Roi_Result_1074702561/202101051415_EPHYS_NWB_STIMULUS_SUMMARY_V3_QUEUE_1074702561_output.json
 
{
      "sweep_number": 5,
      "stimulus_units": "Amps",
      "bridge_balance_mohm": 10.98901081085205,
      "leak_pa": null,
      "stimulus_scale_factor": 10.0,
      "stimulus_code": "X1PS_SubThresh",
      "stimulus_code_ext": "X1PS_SubThresh[1]",
      "clamp_mode": "CurrentClamp",
      "stimulus_name": "Long Square",
      "tags": [
        "Recording stopped before completing the experiment epoch"
      ],
      "stimulus_start_time": 1.5e-05,
      "stimulus_amplitude": null,   <-------------------------------------------------------------------------------------------------
      "stimulus_duration": 3.59998,
      "stimulus_interval": null
    },

[t-b]

What Rusty says makes totally sense. Instead of looking into the acquired AD data, you could also look into the stimulus sets to get the amplitude. This works will all MIES versions. Depending on the stimulus set you could also try out the new epoch information, (available from AllenInstitute/MIES@6191bae on) which is available in the labnotebook. It holds various properties of the DA wave as key value pairs.

[sgratiy]

I assume this concerns only current clamp sweeps as in example above.
In this case we can address the issue by looking into

tags = check_sweep_integrity(sweep, is_ramp)
in qc_feature_extractor.py#L280

This function is checking for the presence of all the define epochs.
For the null stimulus_amplitude we could make stimulus epoch computed by
get_stim_epoch(i, test_pulse=True)
to return None that should fail the sweep.