Advance the json output option to produce a valid empty json in case a scanned target is not reachable.

[manuel-sommer]

drHEADer version: 1.7.0

drHEADer should output a valid JSON file which is empty in case a target can't be evaluated (e.g. if not reachable).
At the moment no JSON output is produced in such a case.

Summarizing, the option "--json" gives inconsistent output in various scenarios.

[emilejq]

I disagree with this. If a target isn't reachable or can't be scanned for some reason, then an error has occurred and that should be fed back to the caller

Happy to review what gets fed back to the caller if you think it's not user-friendly at the moment, but returning an empty JSON in the event of a failure would give no information to the user about the error, and would make it indistinguishable from a scan that ran successfully but didn't find any issues

What other scenarios have you encountered giving inconsistent output?

[rumaf]

@emilejq Current implementation breaks the bulk scan report if one of the targets can't be evaluated (e.g., if it's not reachable). When one of the targets encounters an error, it results in an empty file (not valid JSON) being created.

A possible workaround could be to consistently include errors in the report when they occur, for example:

[
  {
    "url": "https://example.com",
    "report": [
      {
        "message": "Failed to establish a new connection: nodename nor servname provided, or not known",
        "severity": "warn"
      }
    ]
  },
  {
    "url": "https://example2.com",
    "report": [
      {
        "rule": "Cache-Control",
        "message": "Value does not match security policy",
        "severity": "high",
        "value": "no-cache",
        "expected": [
          "no-store",
          "max-age=0"
        ],
        "delimiter": ","
      }
    ]
  }
]

[emilejq]

Hi both,

Thanks for reporting this bug!

This has now been fixed and released in version 2.0.0. Bulk scans now return an empty report and an error for a failing target without breaking the scan for other targets.

Note that the CLI option for JSON output has changed in v2 from --json to --output json

[
    {
        "url": "https://iojfoeirjgoiderjgodijfgdofijg.net",
        "report": [],
        "error": "HTTPSConnectionPool(host='iojfoeirjgoiderjgodijfgdofijg.net', port=443): Max retries exceeded with url: / (Caused by NameResolutionError(\"<urllib3.connection.HTTPSConnection object at 0x1065e1390>: Failed to resolve 'iojfoeirjgoiderjgodijfgdofijg.net' ([Errno 8] nodename nor servname provided, or not known)\"))"
    },
    {
        "url": "https://example.org",
        "report": [
            {
                "rule": "Cache-Control",
                "message": "Value does not match security policy",
                "severity": "high",
                "value": "max-age=604800",
                "expected": [
                    "no-store",
                    "max-age=0"
                ],
                "delimiter": ","
            },
            {
                "rule": "Content-Security-Policy",
                "message": "Header not included in response",
                "severity": "high"
            }
        ]
    }
]