Greatings fellow Pythonistas and Emacs users!
Have you ever worked on a project that uses one of the many Python package managers and/or virtual environments, where all the linters, formatters and commit hooks are set up meticulously, and then when you fire up Emacs, packages like flycheck or lsp-mode are either unable to find the binary in your virtualenv, or are using the wrong one?
Have you ever tried one of the 11+ Emacs virtualenv packages to help you fix this problem, but are still at a lost at why your other favorite Emacs packages still can't find the right binaries, or they stop working when you switch to a different project using a different flavor of virtualenv?
If you answer "yes" for any of these questions, you've come to the right place.
The first key insight is to recognize the paths to the executables of many Python linting and formatting Emacs packages are configurable.
The second key insight is Emacs allows you to setup a different value for the exectuable path on a per buffer basis, and that these packages work with these buffer-local values.
The hardest problem is finding the correct executable, this is what pet
tries to solve.
As long as you use one of the supported Python virtualenv tools, pet will be
able to find the virtualenv root and binary you ask for, with zero Emacs
configuration necessary.
pet works well with popular source code project management packages such as
Projectile and the
built-in project.el. The first time you call one the few pet helper
functions, it will use Projectile or project.el to detect the root of your
project, search for the configuration files for the many supported Python
virtualenv tools, and then lookup the location of the virtualenv based on the
content of the configuration files. Once a virtualenv is found, all executables
are found by looking into its bin directory.
- pre-commit
- poetry
- pipenv
- direnv
- venv, virtualenv or virtualenvwrapper (virtualenvwrapper caveats)
- maturin
- uv (but not tools installed by
uv tool install) - pdm
- pipx
- pyenv (very poorly maintained, don't use it unless you are using Homebrew on macOS)
- docker
- conda
- mamba
- micromamba
- Whatever is on your
VIRTUAL_ENVenvironment variable - Even when you aren't in a virtual environment
- Built-in project.el
- projectile
- envrc (direnv caveats)
- eglot
- dape
- flycheck
- lsp-jedi
- lsp-pyright
- dap-python
- blacken
- yapfify
- python-black
- python-isort
- python-pytest
- ruff-format
- py-autopep8
- auto-virtualenvwrapper
Currently pet requires a program to convert TOML to JSON, a program to
convert YAML to JSON, and if you are using Emacs < 29, the sqlite3 command
to be installed on your system.
By default, both the TOML to JSON and YAML to JSON converters are configured to use dasel. If you are on Linux, it may be more convenient to use tomljson and yq since both of which are likely to be available from the system package management system.
When a suitable Emacs Lisp YAML and TOML parser becomes available, dasel
will be made optional.
If you are using Emacs on macOS, to get the most out of pet, it is best to
install exec-path-from-shell first to ensure all of the
Supported Python Virtual Environment Tools are available in your
exec-path. Once your exec-path is synced up to your shell's $PATH
environment variable, you can use the following ways to help you setup the rest
of your Emacs packages properly.
Generally, the following snippet is all you'll need:
(require 'pet)
;; Emacs < 26
;; You have to make sure this function is added to the hook last so it's
;; called first
(add-hook 'python-mode-hook 'pet-mode)
;; Emacs 27+
;; The -10 tells `add-hook' to makes sure the function is called as early as
;; possible whenever it is added to the hook variable
(add-hook 'python-mode-hook 'pet-mode -10)
;; Emacs 29+
;; This will turn on `pet-mode' on `python-mode' and `python-ts-mode'
(add-hook 'python-base-mode-hook 'pet-mode -10)Or, if you use use-package:
(use-package pet
:config
(add-hook 'python-base-mode-hook 'pet-mode -10))This will setup the buffer local variables for all of the Supported Emacs Packages.
If you need to configure a package that pet doesn't support, or only want to
configure a couple of packages instead of all of the supported ones, pet
offers 2 autoloaded functions to help you find the correct path to the
executable and virtualenv directory:
(pet-executable-find EXECUTABLE)(pet-virtualenv-root)
For example, to set up python-mode to use the correct interpreter when you
execute M-x run-python:
(add-hook 'python-mode-hook
(lambda ()
(setq-local python-shell-interpreter (pet-executable-find "python")
python-shell-virtualenv-root (pet-virtualenv-root))))For flycheck, due to its complexity, pet also comes with another
autoloaded function to help you setup the flake8, pylint and mypy
checkers:
(add-hook 'python-mode-hook 'pet-flycheck-setup)(use-package exec-path-from-shell
:if (memq (window-system) '(mac ns))
:config (exec-path-from-shell-initialize))
(use-package flycheck)
(use-package lsp)
(use-package lsp-jedi
:after lsp)
(use-package lsp-pyright
:after lsp)
(use-package dap-python
:after lsp)
(use-package eglot)
(use-package python-pytest)
(use-package python-black)
(use-package python-isort)
(use-package ruff-format)
(use-package pet
:ensure-system-package (dasel sqlite3)
:config
(add-hook 'python-mode-hook
(lambda ()
(setq-local python-shell-interpreter (pet-executable-find "python")
python-shell-virtualenv-root (pet-virtualenv-root))
;; (pet-eglot-setup)
;; (eglot-ensure)
(pet-flycheck-setup)
(flycheck-mode)
(setq-local lsp-jedi-executable-command
(pet-executable-find "jedi-language-server"))
(setq-local lsp-pyright-python-executable-cmd python-shell-interpreter
lsp-pyright-venv-path python-shell-virtualenv-root)
(lsp)
(setq-local dap-python-executable python-shell-interpreter)
(setq-local python-pytest-executable (pet-executable-find "pytest"))
(when-let ((ruff-executable (pet-executable-find "ruff")))
(setq-local ruff-format-command ruff-executable)
(ruff-format-on-save-mode))
(when-let ((black-executable (pet-executable-find "black")))
(setq-local python-black-command black-executable)
(python-black-on-save-mode))
(when-let ((isort-executable (pet-executable-find "isort")))
(setq-local python-isort-command isort-executable)
(python-isort-on-save-mode)))))Short answer:
Use envrc.
(require 'envrc)
(add-hook 'change-major-mode-after-body-hook 'envrc-mode)Longer answer:
There are a number of packages similar to envrc such as direnv and
buffer-env that claim to be able to configure direnv in Emacs. However,
they all suffer from various problems such as changing the environment and
exec-path for the entire Emacs process, unable to activate early enough or
being too general to support direnv tightly.
Because pet needs to configure the buffer local variables before the
rest of the minor modes are activated, but after exec-path has been set
up by direnv, one must take care of choosing a minor mode package that allows
the user to customize when it takes effect. This requirement rules out
direnv.el [1].
| [1] | Earlier versions of pet suggested direnv.el as a solution, it is
no longer recommended due to this reason. |
You can use envrc + this direnv configuration to activate
your virtualenv or auto-virtualenvwrapper. Note that in
any case, your virtualenv must be activated before turning on pet-mode in
order to make the environment variable VIRTUAL_ENV available to it. For
example:
(require 'auto-virtualenvwrapper)
(require 'pet)
(add-hook 'python-base-mode-hook
(lambda ()
(auto-virtualenvwrapper-activate)
(pet-mode))
-10)
(add-hook 'window-configuration-change-hook #'auto-virtualenvwrapper-activate)
(add-hook 'focus-in-hook #'auto-virtualenvwrapper-activate)Pet does not automatically create virtualenvs for you. If you have a fresh
clone, you must create the virtualenv and install your development dependencies
into it first. Once it is done, the next time you open a Python file buffer
pet will automatically set up the executable variables for you.
To find out how to do it, please find the virtualenv tool in question from Supported Python Virtual Environment Tools, and visit its documentation for details.
The reason is mainly due to the fact that many Python projects use development
tools located in different virtualenvs. This means exec-path needs to be
prepended with all of the virtualenvs for all of the dev tools, and always kept
in the correct order. An example where this approach may cause issues is dealing
with projects that use pre-commit and direnv. A typical pre-commit
configuration may include many "hooks", where each of them is isolated in its
own virtualenv. While prepending many directories to exec-path is not
problematic in itself, playing well with other Emacs packages that mutate
exec-path reliably is non-trivial. Providing an absolute path to executable
variables conveniently sidesteps this complexity, while being slightly more
performant.
In addition, there are Emacs packages, most prominantly flycheck that by
default require dev tools to be installed into the same virtualenv as the first
python executable found on exec-path. Changing this behavior requires
setting the corresponding flycheck checker executable variable to the
intended absolute path.
You can turn on pet-debug and watch what comes out in the *Messages*
buffer. In addition, you can use M-x pet-verify-setup in your Python buffers
to find out what was detected.
For lsp, use lsp-describe-session.
For eglot, use eglot-show-workspace-configuration.
For flycheck, use flycheck-verify-setup.
Nope. You can uninstall them all. This is the raison d'être of this package.