pipdeptree is a command line utility for displaying the installed
python packages in form of a dependency tree. It works for packages
installed globally on a machine as well as in a virtualenv. Since
pip freeze shows all dependencies as a flat list, finding out
which are the top level packages and which packages do they depend on
requires some effort. It's also tedious to resolve conflicting
dependencies that could have been installed because older version of
pip didn't have true dependency resolution [1]. pipdeptree
can help here by identifying conflicting dependencies installed in the
environment.
To some extent, pipdeptree is inspired by the lein deps :tree
command of Leiningen.
$ pip install pipdeptreepipdeptree has been tested with Python versions 2.7, 3.5,
3.6, 3.7, 3.8, 3.9 as well as pypy2 and pypy3.
Python 2.6 is way past it's end of life but if you ever find
yourself stuck on a legacy environment, version 0.9.0 might
work.
New in ver. 2.0.0
If you want to run pipdeptree in the context of a particular
virtualenv, you can specify the --python option. Note that this
capability has been recently added in version 2.0.0.
Alternately, you may also install pipdeptree inside the virtualenv and then run it from there.
To give you a brief idea, here is the output of pipdeptree
compared with pip freeze:
$ pip freeze
Flask==0.10.1
itsdangerous==0.24
Jinja2==2.11.2
-e git+git@github.com:naiquevin/lookupy.git@cdbe30c160e1c29802df75e145ea4ad903c05386#egg=Lookupy
MarkupSafe==0.22
pipdeptree @ file:///private/tmp/pipdeptree-2.0.0b1-py3-none-any.whl
Werkzeug==0.11.2And now see what pipdeptree outputs,
$ pipdeptree
Warning!!! Possibly conflicting dependencies found:
* Jinja2==2.11.2
- MarkupSafe [required: >=0.23, installed: 0.22]
------------------------------------------------------------------------
Flask==0.10.1
- itsdangerous [required: >=0.21, installed: 0.24]
- Jinja2 [required: >=2.4, installed: 2.11.2]
- MarkupSafe [required: >=0.23, installed: 0.22]
- Werkzeug [required: >=0.7, installed: 0.11.2]
Lookupy==0.1
pipdeptree==2.0.0b1
- pip [required: >=6.0.0, installed: 20.1.1]
setuptools==47.1.1
wheel==0.34.2New in ver. 0.5.0
Yes, there's a --reverse (or simply -r) flag for this. To find
out which packages depend on a particular package(s), it can be
combined with --packages option as follows:
$ pipdeptree --reverse --packages itsdangerous,MarkupSafe
Warning!!! Possibly conflicting dependencies found:
* Jinja2==2.11.2
- MarkupSafe [required: >=0.23, installed: 0.22]
------------------------------------------------------------------------
itsdangerous==0.24
- Flask==0.10.1 [requires: itsdangerous>=0.21]
MarkupSafe==0.22
- Jinja2==2.11.2 [requires: MarkupSafe>=0.23]
- Flask==0.10.1 [requires: Jinja2>=2.4]As seen in the above output, pipdeptree by default warns about
possible conflicting dependencies. Any package that's specified as a
dependency of multiple packages with different versions is considered
as a conflicting dependency. Conflicting dependencies are possible if
older version of pip<=20.2 (without the new resolver [1]) was ever used to
install dependencies at some point. The warning is printed to stderr
instead of stdout and it can be completely silenced by specifying the
-w silence or --warn silence option. On the other hand, it can
be made mode strict with --warn fail, in which case the command
will not only print the warnings to stderr but also exit with a
non-zero status code. This is useful if you want to fit this tool into
your CI pipeline.
Note: The --warn option is added in version 0.6.0. If you
are using an older version, use --nowarn flag to silence the
warnings.
In case any of the packages have circular dependencies (eg. package A
depends on package B and package B depends on package A), then
pipdeptree will print warnings about that as well.
$ pipdeptree --exclude pip,pipdeptree,setuptools,wheel
Warning!!! Cyclic dependencies found:
- CircularDependencyA => CircularDependencyB => CircularDependencyA
- CircularDependencyB => CircularDependencyA => CircularDependencyB
------------------------------------------------------------------------
wsgiref==0.1.2
argparse==1.2.1Similar to the warnings about conflicting dependencies, these too are
printed to stderr and can be controlled using the --warn option.
In the above example, you can also see --exclude option which is
the opposite of --packages ie. these packages will be excluded
from the output.
If you wish to track only top level packages in your
requirements.txt file, it's possible by grep-ing [2]. only the
top-level lines from the output,
$ pipdeptree --warn silence | grep -E '^\w+'
Flask==0.10.1
gnureadline==8.0.0
Lookupy==0.1
pipdeptree==2.0.0b1
setuptools==47.1.1
wheel==0.34.2There is a problem here though - The output doesn't mention anything
about Lookupy being installed as an editable package (refer to
the output of pip freeze above) and information about its source
is lost. To fix this, pipdeptree must be run with a -f or
--freeze flag.
$ pipdeptree -f --warn silence | grep -E '^[a-zA-Z0-9\-]+'
Flask==0.10.1
gnureadline==8.0.0
-e git+git@github.com:naiquevin/lookupy.git@cdbe30c160e1c29802df75e145ea4ad903c05386#egg=Lookupy
pipdeptree @ file:///private/tmp/pipdeptree-2.0.0b1-py3-none-any.whl
setuptools==47.1.1
wheel==0.34.2
$ pipdeptree -f --warn silence | grep -E '^[a-zA-Z0-9\-]+' > requirements.txtThe freeze flag will not prefix child dependencies with hyphens, so
you could dump the entire output of pipdeptree -f to the
requirements.txt file thus making it human-friendly (due to
indentations) as well as pip-friendly.
$ pipdeptree -f | tee locked-requirements.txt
Flask==0.10.1
itsdangerous==0.24
Jinja2==2.11.2
MarkupSafe==0.23
Werkzeug==0.11.2
gnureadline==8.0.0
-e git+git@github.com:naiquevin/lookupy.git@cdbe30c160e1c29802df75e145ea4ad903c05386#egg=Lookupy
pipdeptree @ file:///private/tmp/pipdeptree-2.0.0b1-py3-none-any.whl
pip==20.1.1
setuptools==47.1.1
wheel==0.34.2On confirming that there are no conflicting dependencies, you can even
treat this as a "lock file" where all packages, including the
transient dependencies will be pinned to their currently installed
versions. Note that the locked-requirements.txt file could end up
with duplicate entries. Although pip install wouldn't complain
about that, you can avoid duplicate lines (at the cost of losing
indentation) as follows,
$ pipdeptree -f | sed 's/ //g' | sort -u > locked-requirements.txtNew in ver. 0.5.0
It's also possible to have pipdeptree output json representation
of the dependency tree so that it may be used as input to other
external tools.
$ pipdeptree --jsonNote that --json will output a flat list of all packages with
their immediate dependencies. This is not very useful in itself. To
obtain nested json, use --json-tree
New in ver. 0.11.0
$ pipdeptree --json-tree
The dependency graph can also be visualized using GraphViz:
$ pipdeptree --graph-output dot > dependencies.dot
$ pipdeptree --graph-output pdf > dependencies.pdf
$ pipdeptree --graph-output png > dependencies.png
$ pipdeptree --graph-output svg > dependencies.svgNote that graphviz is an optional dependency ie. required only if
you want to use --graph-output. If the version of graphviz
installed in the env is older than 0.18.1, then a warning will be
displayed about upgrading graphviz. Support for older versions of
graphviz will be dropped soon.
Since version 2.0.0b1, --package and --reverse flags are
supported for all output formats ie. text, json, json-tree and graph.
In earlier versions, --json, --json-tree and
--graph-output options override --package and --reverse.
usage: pipdeptree.py [-h] [-v] [-f] [--python PYTHON] [-a] [-l] [-u]
[-w [{silence,suppress,fail}]] [-r] [-p PACKAGES]
[-e PACKAGES] [-j] [--json-tree]
[--graph-output OUTPUT_FORMAT]
Dependency tree of the installed python packages
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-f, --freeze Print names so as to write freeze files
--python PYTHON Python to use to look for packages in it (default:
where installed)
-a, --all list all deps at top level
-l, --local-only If in a virtualenv that has global access do not show
globally installed packages
-u, --user-only Only show installations in the user site dir
-w [{silence,suppress,fail}], --warn [{silence,suppress,fail}]
Warning control. "suppress" will show warnings but
return 0 whether or not they are present. "silence"
will not show warnings at all and always return 0.
"fail" will show warnings and return 1 if any are
present. The default is "suppress".
-r, --reverse Shows the dependency tree in the reverse fashion ie.
the sub-dependencies are listed with the list of
packages that need them under them.
-p PACKAGES, --packages PACKAGES
Comma separated list of select packages to show in the
output. If set, --all will be ignored.
-e PACKAGES, --exclude PACKAGES
Comma separated list of select packages to exclude
from the output. If set, --all will be ignored.
-j, --json Display dependency tree as json. This will yield "raw"
output that may be used by external tools. This option
overrides all other options.
--json-tree Display dependency tree as json which is nested the
same way as the plain text output printed by default.
This option overrides all other options (except
--json).
--graph-output OUTPUT_FORMAT
Print a dependency graph in the specified output
format. Available are all formats supported by
GraphViz, e.g.: dot, jpeg, pdf, png, svgpipdeptreerelies on the internal API ofpip. I fully understand that it's a bad idea but it mostly works! On rare occasions, it breaks when a new version ofpipis out with backward incompatible changes in internal API. So beware if you are using this tool in environments in whichpipversion is unpinned, specially automation or CD/CI pipelines.
pipdeptree merely looks at the installed packages in the current
environment using pip, constructs the tree, then outputs it in the
specified format. If you want to generate the dependency tree without
installing the packages, then you need a dependency resolver. You
might want to check alternatives such as pipgrip or poetry.
There are 2 test suites in this repo:
- Unit tests that use mock objects. These are configured to run on every push to the repo and on every PR thanks to Github Actions.
- End-to-end tests that are run against actual packages installed in virtualenvs
Unit tests can be run against all version of python using tox as follows:
$ make test-tox-allThis assumes that you have python versions specified in the
tox.ini file.
If you don't want to install all the versions of python but want to
run tests quickly against Python3.6 only:
$ make testUnit tests are written using pytest and you can also run the tests
with code coverage as follows,
$ make test-covOn the other hand, end-to-end tests actually create virtualenvs,
install packages and then run tests against them. These tests are more
reliable in the sense that they also test pipdeptree with the
latest version of pip and setuptools.
The downside is that when new versions of pip or setuptools
are released, these need to be updated. At present the process is
manual but I have plans to setup nightly builds for these for faster
feedback.
The end-to-end tests can be run as follows,
$ make test-e2e # starts with a clean virtualenvs
$ # or
$ make test-e2e-quick # reuses existing virtualenvsBy default the e2e tests uses python executable python3.6. To use
an alternate version set the environment var E2E_PYTHON_EXE.
$ E2E_PYTHON_EXE=python2.7 make test-e2e- Make sure that tests pass on Github Actions.
- Create a commit with following changes and push it to github
- Update the __version__ in the pipdeptree.py file.
- Add Changelog in CHANGES.md file.
- Also update README.md if required.
- Create an annotated tag on the above commit and push the tag to github
- Upload new version to PyPI.
MIT (See LICENSE)
| [1] | (1, 2) pip version 20.3 has been released in Nov 2020 with the dependency resolver <https://blog.python.org/2020/11/pip-20-3-release-new-resolver.html>_ |
| [2] | If you are on windows (powershell) you can run
pipdeptree --warn silence | Select-String -Pattern '^\w+'
instead of grep |