ioos/compliance-checker

skip-checks option doesn't work as documented

mhidas opened this issue · 0 comments

Version of compliance checker running: 5.x

Describe the checker this affects:
Core checker functionality, i.e., affects any check suite.

Attach a minimal CDL or NetCDF file which is able to reproduce the issue
Any file that fails a specific check method. e.g. this file is missing the recommended history attribute, so is fails the check_convention_globals check in the CF check suite.

To Reproduce:
For the example file above, if I want to ignore that specific CF recommendation and have the checker pass the file, I could skip the check_convention_globals. The help text from the command-line interface describes this option:

  --skip-checks SKIP_CHECKS, -s SKIP_CHECKS
                        Specifies tests to skip. Can take the form of either `<check_name>` or
                        `<check_name>:<skip_level>`. The first form skips any checks matching the
                        name. In the second form <skip_level> may be specified as "A", "M", or
                        "L". "A" skips all checks and is equivalent to calling the first form. "M"
                        will only show high priority output from the given check and will skip
                        medium and low. ...

However, the simple <check_name> form doesn't work anymore (it did with version 4.x):

$ compliance-checker -t cf:1.6 -s check_convention_globals IMOS_ANMN-AM_GST_20190419T100000Z_NRSMAI_FV01_NRSMAI-CO2-1904-delayed_END_20191002T040000Z_C-20210525T140027Z.nc 

...
                                    Warnings                                    
--------------------------------------------------------------------------------
§2.6 Attributes
* §2.6.2 global attribute history should exist and be a non-empty string

Describe the issue below:
Specifying the second form of the option (-s check_convention_globals:M), does skip the check as expected.
For context, we rely on this functionality in cases where it is not possible/practical to make a file fully compliant, or there is a bug in the checker that falsely rejects certain files, but we want to apply the rest of the checks to all files passing through our data ingenstion pipelines.