Bug: licensecheck relies on Pypi even if private registries are declared in Pyproject.toml

[Azraeht]

Bug

System info

• OS: Ubuntu 22.04
• Version: 2023.4.2

Describe the bug

Here is a sample (simplified) pyproject.toml

[build-system]
requires = [ "poetry>=0.12",]
build-backend = "poetry.masonry.api"

[tool.poetry]
name = "my-project"
version = "0.0.0"
description = "My Amazing Project"
authors = [ "Brice Santus",]
license = "MIT"

[[tool.poetry.source]]
name = "devpi"
url = "https://private-devpi.custom-domain.com/root/+simple/"
default = true

[tool.poetry.dependencies]
python = "~3.9"
flask = "^2"
deduplication = "==1.2.1"

In such case we are using 2 dep: flask and deduplication. In our dev context both are retrieved through Devpi (or any mirroring private registry). flask is the well known public dep while deduplication is here a custom internal lib stored in our devpi instance.

While licensecheck will run it will try to retrieve data for deduplication from https://pypi.org/ and will find https://pypi.org/project/deduplication/ which is completely unrelated with our own lib.
This behaviour will cause licensecheck to retrieve a wrong information. (And in this particular case, to crash because https://pypi.org/project/deduplication/ is not well register and its metadata can't be well parser by packaging builtin lib)

I'm trying to find the best way to solve it and I would like your advices. Two options:

• Implement support of custom registry for remote check of dependencies: not so trivial depending on how each registry could retrieve packages inof
• Make evolve the ignorePackages flag to completely skip package processing and not only licence checking on them

For both options I can work on a Pull Request.

What do you think?

Expected outcome

We expect options on licensecheck to handle private dependencies stored in private registries

Actual outcome

licensecheck always rely on Pypi even when parsing private libs

[FredHappyface]

Hi thanks for creating this issue

I think for now the second option would be preferable. Certainly happy looking at a pr - thanks for your time 😊

There might be some additional work from me to work on the stability for when parsing fails

[Azraeht]

Thanks for merging!
Would you mind to release a 2023.4.3 tag with this fix? Thanks by advance