Missing dependency due to default --max-bacon 2

[terwilliger42]

Hi @thebjorn - thanks for all of your hard work on this project! Your tool is extremely helpful the work that I'm doing. I am so grateful. 👍

That said, I'm running into a little trouble. I've simplified into a very basic testcase:

test:
  bar:
    __init__.py: print("imported bar")
  foo:
    __init__.py: |-
      import bar
      print("Imported foo")
  run_test.py: import foo

When I run this, I don't get a "bar" dependency from pydeps, even though it is being imported.

$ cd test
$ ../py38/bin/python run_test.py
imported bar
Imported foo
$ 
../py38/bin/pydeps run_test.py 

Expect "bar" dependency on "foo"

Any idea what the cause is? Thanks again for all of your work on this project.

[terwilliger42]

Oh, I see the issue. bar is being excluded due to the default --max-bacon 2. It took me a while stepping through the debugger to figure this out.

I suggest a default --max-bacon 0 i.e. by default show all dependencies, then users can filter out data if it's too much -- rather than quietly masking data by default. Still grateful for the tool. Thanks so much. 😄 👍

[thebjorn]

Hi @terwilliger42 and thank you for your interest in pydeps, and especially thank you for your feedback.

iirc, the max-bacon used to be a lot higher, but the resulting graphs were so dense that they were uninformative. That was before grouping, path-highlighting, and automatic stdlib removal was added though, so perhaps it is time to revisit the issue... ;-)

[terwilliger42]

Yes that makes a lot sense. 👍

For our use case, we're not using the graphic visualization features at all. Our goal is to verify that we aren't accidentally importing code that we're not supposed to, i.e.

1. Write a CI check that uses pydeps JSON output to build a dependency list & verify only supported modules are imported.
2. If some module is being imported that is not allowed, show where the import occurred.

For (2), the plan is to load the pydeps JSON output into networkx (pypi package for network/graph structures) and then calculate shortest path from main to the problem module. (i.e. print a string like "foo -> bar -> baz").

[thebjorn]

Sounds interesting :-)

A call like pydeps yourapp --max-bacon 0 --no-show --show-raw-deps will give you everything (--show-deps instead of --show-raw-deps).

You should be able to just loop over the top level of the json looking for non-supported models, and then look at the "imported_by" subkey to find where it is being imported from, e.g.:

"urllib3.util.retry": {
    "bacon": 8,
    "imported_by": [
        "requests.adapters",
        "urllib3",
        "urllib3.connectionpool",
        "urllib3.poolmanager",
        "urllib3.util"
    ],
    "imports": [
        "urllib3.exceptions",
        "urllib3.packages",
        "urllib3.packages.six"
    ],
    "name": "urllib3.util.retry",
    "path": "c:\\srv\\venv\\dev35\\lib\\site-packages\\urllib3\\util\\retry.py"
},