PyO3/pyo3

PyO3 Fails to Import DLL in Python Virtual Environment on Windows

luoshuijs opened this issue · 9 comments

luoshuijs commented

Bug Description

When running cargo test within a python virtual environment on Windows, PyO3 fails to import DLLs, resulting in a STATUS_DLL_NOT_FOUND error. This occurs even after attempting to use --no-default-features to exclude potential issues caused by default feature sets.

Additional Attempt to Resolve:
Following the suggestion from the PyO3 FAQ, adding the directory containing the Python DLL (from the existing Python environment) to the PATH environment variable was attempted. However, this leads to importing modules from the existing Python environment instead of the virtual environment.

Steps to Reproduce

  1. Set up a python virtual environment on Windows.

  2. Create a minimal Rust project using PyO3.

  3. Add a simple test case in src/lib.rs, for example:

    #[test]
    fn test() {
        pyo3::prepare_freethreaded_python();
        Python::with_gil(|py| {
            let _numpy = PyModule::import(py, "numpy").unwrap();
        });
    }

  1. Run the project using cargo test.

  2. Observe the error output.

Backtrace

No response

Your operating system and version

Win 10

Your Python version (python --version)

Python 3.11.0

Your Rust version (rustc --version)

rustc 1.71.0-nightly (4a59ba4d5 2023-05-12)

Your PyO3 version

0.19.2

How did you install python? Did you use a virtualenv?

python.org install
python -v venv venv

Additional Info

No response

luoshuijs commented

I found that this issue is similar to #1896. I also found that the above steps work normally on Linux (with Ubuntu 22.1 as the system) when running tests with --no-default-features as a parameter. Could this possibly be an issue unique to Windows systems and Python?

davidhewitt commented

This is almost certainly a duplicate of #1896. It's on my list to solve eventually, I think it'll get easier when we can drop 3.7 support and adopt the new PyConfig APIs. @alex - how is the cryptography situation with 3.7 looking? In pydantic we will be dropping 3.7 in our 2.6 release (so probably by end of year).

@luoshuijs, for now you should be able to workaround this by using PYTHONHOME and PYTHONPATH environment variables.

alex commented

Quite bad :-( It's still something like 1/4 of all of our downloads. From the metrics, a huge portion of the downloads comes from Amazon Linux 2.

  • 👍1
  • 👀1
luoshuijs commented

Thank you very much, @davidhewitt, for your response. However, after trying to set the PYTHONHOME and PYTHONPATH environment variables as you suggested, I'm still facing issues. Here's what I did in PowerShell:.

PS K:\PycharmProjects\python_genshin_artifact> venv/Scripts/activate.ps1
(venv) PS K:\PycharmProjects\python_genshin_artifact>
(venv) PS K:\PycharmProjects\python_genshin_artifact> $env:PYTHONHOME="C:\Users\luoshuijs\AppData\Local\Programs\Python\Python311"
(venv) PS K:\PycharmProjects\python_genshin_artifact> $env:PYTHONPATH="C:\Users\luoshuijs\AppData\Local\Programs\Python\Python311"
(venv) PS K:\PycharmProjects\python_genshin_artifact> $env:PYTHONHOME                                                             
C:\Users\luoshuijs\AppData\Local\Programs\Python\Python311
(venv) PS K:\PycharmProjects\python_genshin_artifact> $env:PYTHONPATH                                                             
C:\Users\luoshuijs\AppData\Local\Programs\Python\Python311
(venv) PS K:\PycharmProjects\python_genshin_artifact> cargo test                                                                  
    Finished test [unoptimized + debuginfo] target(s) in 0.27s
     Running unittests src\lib.rs (target\debug\deps\_python_genshin_artifact-9ae2f0247525dfea.exe)
error: test failed, to rerun pass `--lib`

Caused by:
  process didn't exit successfully: `K:\PycharmProjects\python_genshin_artifact\target\debug\deps\_python_genshin_artifact-9ae2f0247525dfea.exe` (exit code: 0xc0000135, STATUS_DLL_NOT_FOUND)
davidhewitt commented

@luoshuijs did you sat PATH too? I think you'll still need that also.

  • 👍1
luoshuijs commented

@luoshuijs did you sat PATH too? I think you'll still need that also.

Indeed, I forgot to set the Path. After making the modification, it did have an effect. However, the corresponding modules were imported from the existing Python environment (not a virtual environment), but it's not a big issue. I plan to write a script to resolve the above issue. I am very grateful for your response. I hope to completely fix this problem by the end of this year.