AllenInstitute/ipfx

add ability to set specific sweep checks as optional

Opened this issue · 2 comments

Story:
As a scientist/analyst processing non-IVSCC data (internal or external), I want to be able to selectively drop QC checks rather than just loosen bounds, in case a test sweep is not present in my recording.

Repro:

python -m ipfx.bin.run_pipeline_from_nwb_file /allen/aibs/technology/waynew/ephys/chartrand/H20.29.177.11.11.01.nwb /allen/aibs/technology/waynew/ephys/chartrand/output/

Result:
Fails for electrode_0, seal, and input_access_resistance

failed_blowout:False
failed_electrode_0:True
failed_seal:True
failed_input_access_resistance:True
fail_tags:['electrode_0_pa missing value', 'Invalid seal (None)', 'Resistance ratio not available', 'Initial access resistance not available']

Expected solution:
Provide a way to define which of these checks to use, so that checks not specified will have no entry in cell_fail_tags and the relevant cell_state entry would be False or missing

Note that the blowout check already ignores missing values by default

Our thinking (3/2/2021) to address this issue is to make these checks to be parameters. Check the list against QC criteria to exclude particular checks.

Output should include which criteria were checked for QC.

May require a change to input parameters schema.

We believe this is all outside of the LIMS pipeline (for now, at least) so no LIMS work needed.

I believe it would be better to make QC criteria optional. Only criteria provided, would be used for QC checks.
Then we would pass QC criteria as parameters to the ipfx.bin.run_pipeline_from_nwb_file

This would require:

  • updating QcCriteria schema
  • adding argument to the ipfx.bin.run_pipeline_from_nwb_file
  • change to selectively evaluate qc criteria in qc_cell based on available criteria
  • Output to include which criteria were checked for QC.