Return spdx_license_key when fetching license data from LicenseDB or DJE

[DennisClark]

When the about gen process does a fetch of license data from LicenseDB or DJE, it should always return the spdx_license_key (when available) and it should write that field to the .ABOUT file when it creates it.

The about inventory process should also write the spdx_license_key to the inventory output file.

[chinyeungli]

we need to also update the spec (and code) to include support for spdx_license_key (include validation and stuff)

[DennisClark]

if that's going to take a while, it can possibly wait for v7.0.1

[chinyeungli]

I am thinking instead of having support spdx_license_key natively, perhaps an option in gen to generate spdx_license_key instead? The reason for that is if we say we support spdx_license_key, users may only include the spdx_license_key field in an inventory and not the license_expression field which our current code will not work as it requires the scancode license key to get the spdx_license_key and not the other way around.
@DennisClark what do you think?

[DennisClark]

it seems that you basically stating what I suggested, but making it an option. I don't think we need to do that. Consider that we already get this data using, for example, using
license_expression: bsd-new
we construct this when we write the .ABOUT file:

licenses:
  - key: bsd-new
    name: BSD-3-Clause
    file: bsd-new.LICENSE
    url: https://scancode-licensedb.aboutcode.org/bsd-new.LICENSE

It seems to me that we can add spdx_license_key as one more value to return when we build that information.

[DennisClark]

similarly, when the gen process uses a complex license_expression it could do a similar thing:

license_expression: commercial-license AND mit
would enable it to constuct this:

licenses:
  - key: commercial-license
    name: Commercial License
    file: commercial-license.LICENSE
    url: https://scancode-licensedb.aboutcode.org/commercial-license.LICENSE
  - key: mit
    name: MIT License
    file: mit.LICENSE
    url: https://scancode-licensedb.aboutcode.org/mit.LICENSE

but it would also add a fifth statement on each license to identify the spdx_license_key.

[chinyeungli]

Adding spdx_license_key in ABOUT file can be easily done and it's nice to have. My point is the spdx_license_key is not in the spec and generating something that is not in the spec is a bit strange for me. Perhaps we can document somewhere about the fetch-license will generate spdx_license_key as a bonus a information, but license_expression is still required

[DennisClark]

we can update the spec

[DennisClark]

and yes, of course license_expression is required

[mjherzog]

We should update the spec then. As the SPDX License List gains in popularity with the various SBOM initiatives, it make sense for AbcTK to offer an output option with a complete set of SPDX License Ids or Refs.

[DennisClark]

@chinyeungli I will edit specificatIon.rst to add the new field, and will turn it over to you (please) since I am not all that experienced with the mechanics of updating the GitHub repo. We can ask other folks to review it before making it official.

[DennisClark]

and perhaps, in the spirit of our very short label for the license attributes, we can simply call it spdx_id .

[DennisClark]

but it would be best to stick with the same label that we have in the LicenseDB: spdx_license_key ok -- I will update the spec and provide it for folks to review.

[chinyeungli]

so we update the spec to support spdx_license_key and license_expression is required for fetch-license

now the interesting and the hard part: validation. We need to validate the correctness of the spdx_license_key in either one of the following 2 ways:

• extract the key and validate from spdx.org
• compare the key with DJE (if license_expression is provided)

[DennisClark]

not sure why we need an additional validation step if we are getting the value from LicenseDB or an instance of DJE. What is the reason for an extra validation step?

[chinyeungli]

I agree it's rarely the case that users will create this field by hand, but the validation is in case they do. However, I am open if no validation is needed for this spdx_license_key field, it makes life easier 😄

[DennisClark]

@chinyeungli I am attaching my proposed revision to the specification. Please process it through the correct steps to do a proper pull request. Thanks very much.

specification.rst.zip

[DennisClark]

Note that I suggested a spec version number change to 3.2.2