Add ScanCode LicenseRefs to summary definitions

[elrayle]

Description

Currently, LicenseRefs reported by ScanCode are converted to NOASSERTION or OTHER. Update code to allow LicenseRefs in definitions reported from ClearlyDefined.

NOTE: This may require changes in other repositories as well. Exploration is required to determine where the remapping is happening.

[lumaxis]

I've looked into this a bit and as I understand it today, there's two options for how we can address this. As far as I can tell, the only work required would be in the clearlydefined/service repo.

The place where LicenseRef- expressions get turned into NOASSERTION values is ClearlyDefined's fork of the spdx-expression-parse.js library.
The library calls three sub-functions as part of the parseAtom function: https://github.com/clearlydefined/spdx-expression-parse.js/blob/4d69bb5b95e540da61a4c5c625e204eaedb4eff7/parse.js#L123-L129

function parseAtom () {
  return (
    parseParenthesizedExpression() ||
    parseLicenseRef() ||
    parseLicense()
  )
}

While the parseLicenseRef function does have support for LicenseRef- expressions, what we call this function with is a ScanCode license expression, not yet a SPDX expression.
We do pass a "visitor" function to SPDX.parse, however, this function only gets called in parseLicense which runs _after parseLicenseRef so our function that uses the scancodeMap to turn the ScanCode expression into SPDX runs after the parsing code that supports LicenseRef.

In my opinion, there are two ways for changing the behavior to support LicenseRef- going forward:

1. Update spdx-expression-parse.js to run the visitor function first, before parseLicenseRef is called
2. Update the way we call SPDX.parse so that the use of scancodeMap happens before. I have an experimental commit here that makes this work: lumaxis@1e77eb7

[qtomlinson]

1. Update spdx-expression-parse.js to run the visitor function first, before parseLicenseRef is called

Another variation of 1 can be introducing a licenseRefVisitor similar to licenseVisitor, converting the licenseRef as we parse the licenseRef leaf node.