Editable Installations may be broken in conda environment

[mabruzzo]

Bug report

Editable installations don't seem to work properly in a conda environment.

Historically, I have been able to perform editable installs for other packages in a conda environment (but I'm totally willing to accept that this is a conda-specific issue).

Code for reproduction

I tried this a number of times and I kept encountering problems in various permutations whether I perform pip install -e . or python -m pip install --user -e ..

I just confirmed that if I make a fresh environment I can reproduce this bug.

$ # create and activate a new environment (it doesn't make a difference whether or not I use conda-forge)
$ conda create -c conda-forge -p ~/envs/py310-forge python=3.10
$ conda activate /Users/mabruzzo/envs/py310-forge
# navigate to the yt-directory
$ cd path/to/yt
# do a dummy-install to get all yt dependencies
$ pip install yt
# uninstall yt
$ pip uninstall yt
# now drive a local install (it seems to complete successfully)
$ pip install -v -e .
# test the installation
$ cd ..
$ python -c "import yt; print(yt.load)"

Actual outcome

The final command produces

Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'yt' has no attribute 'load'

Expected outcome

When I use pip install . instead of pip install -e ., then the little snippet prints the following out

$ python -c "import yt; print(yt.load)"
<function load at 0x107c1b5b0>

Version Information

• Operating System: macOS Sonoma 14.4.1
• Python Version: 3.10.14 (packaged by conda-forge)
• yt version: tip of the main branch (commit bc63757)
• Other Libraries (if applicable):

[neutrinoceros]

Hi @mabruzzo, thank you for reporting !
I am very tempted to dismiss this as a conda issue; conda support for editable installs has historically not been great (RIP conda develop), and the fact that using pip is the de-facto (only ?) way to achieve it is worrisome.

A couple questions for you though:

1. Can you explain this ? It looks unnecessary to me since pip install -e . also brings in dependencies

# do a dummy-install to get all yt dependencies
$ pip install yt

2. Have you tried invoking pip as python -m pip instead ? Calling pip directly on the command line can have surprising effects when your path somethow isn't set up properly for your env.

[mabruzzo]

I am very tempted to dismiss this as a conda issue; conda support for editable installs has historically not been great (RIP conda develop),

Yeah, I'm not terribly surprised. In my experience, conda is incredibly frustrating for many things outside of the curated environment (this is probably the impetus for me to switch back to pyenv or give homebrew-python a try). The main reason I reported this in the first place is that I had some luck with editable installs with conda-python in some different projects. But, the circumstances were different enough that I'm totally willing to accept that this is a conda-issue (one case involved python setup.py develop and the other case used scikit-build-core instead of setuptools).

... and the fact that using pip is the de-facto (only ?) way to achieve it is worrisome.

Yeah, as I understand it, at least some of the pypa/pip people don't consider editable installs to be a good practice (but they are obviously convenient). With that said, uv apparently supports editable installs (but I've never actually tried uv),

Can you explain this ? It looks unnecessary to me since pip install -e . also brings in dependencies

# do a dummy-install to get all yt dependencies
$ pip install yt

Yeah, I don't know what I was doing there... It probably isn't necessary. I think I was getting frustrated that various permutations of pip install -e .[full] didn't work, and I picked an alternative path that I knew would work. But I could give it a try if you think it would be helpful.

2. Have you tried invoking pip as python -m pip instead ? Calling pip directly on the command line can have surprising effects when your path somethow isn't set up properly for your env.

Yes, I did try that before I created a fresh conda environment -- and it didn't make a difference compared to pip install -e . . If you think it's worth trying in a fresh environment, I'm happy to do that

[neutrinoceros]

With that said, uv apparently supports editable installs (but I've never actually tried uv)

FWIW I only use uv these days (migrated from pip) and I confirm that it supports editable installs.

About my previous suggestions to try python - pip: creating a brand new environment is pretty cheap, so even if it's likely to fail, I think it's worth trying.

[cphyc]

I already had a similar issue and it was apparently due to Python somehow trying to import yt from the folder yt in the current working folder (rather than the pip-installed package). Could you try cd-ing in another directory that does not contain yt?

[mabruzzo]

I already had a similar issue and it was apparently due to Python somehow trying to import yt from the folder yt in the current working folder (rather than the pip-installed package). Could you try cd-ing in another directory that does not contain yt?

@cphyc -- That totally fixed the problem! Thanks!

In more detail, when I had cloned yt I put it into a folder called ~/packages/py3/yt_dev. At the time, I had a previous, much older, custom-version of the repository called ~/packages/py3/yt that was used as an editable install a while back.

• When I was trying to import yt, I intentionally tried to import if from within ~/packages/py3 (rather than from within ~/packages/py3/yt_dev) to try to avoid this sort of issue.
• But it turns out the existence of this other, unrelated ~/packages/py3/yt directory (frankly, I forgot about it!) was the source of all my problems
• once I renamed ~/packages/py3/yt -> ~/packages/py3/yt_old, all of my problems went away. Things worked as expected

About my previous suggestions to try python - pip: creating a brand new environment is pretty cheap, so even if it's likely to fail, I think it's worth trying.

I also tried this and it doesn't seem to make a large difference. I get consistent results

Can you explain this ? It looks unnecessary to me since pip install -e . also brings in dependencies

I can also confirm that this seems to work. The extra stuff with pip install yt; pip uninstall yt to get dependencies was all user-error.

---

FWIW I only use uv these days (migrated from pip) and I confirm that it supports editable installs.

@neutrinoceros, That's encouraging! I've been tempted to try it out.

I know this is very off topic, but what do you use to manage your python installations? I'm always curious to know what more experienced python developers are doing. (I'm only using conda right now because it was the easiest thing to do to quickly spin up python on a new machine -- previously I had been content with pyenv). The uv readme sounds like it provides pyenv-like support. Do you use that? Or do you do something else?

---

TO SUMMARIZE: This issue was all user-error. We can close this issue

[neutrinoceros]

what do you use to manage your python installations?

At the moment I use pyenv. Up until recently I used pyenv-virtualenv and pip, but replaced both with uv. I'm expecting uv to also make pyenv redundant at some point, as it integrates more and more functionality from rye.

[mabruzzo]

Thanks! That's very insightful!

Closing the issue!