Cannot load pythia_datasets in the Pandas python packages module

Question

Cannot load pythia_datasets in the Pandas python packages module

Closed this issue a year ago · 7 comments

I've set up the Project Pythia environment on my openSUSE Leap 15.4 linux vm with miniconda3 using the following commands:

conda env create --force -f https://raw.githubusercontent.com/ProjectPythia/pythia-foundations/main/environment.yml
conda activate pythia-book-dev

I've gone through most of the modules under https://foundations.projectpythia.org/landing-page.html using spyder, and I'm now trying to do the Pandas module https://foundations.projectpythia.org/core/pandas/pandas.html.

The 1st example starts with the following 2 lines:
import pandas as pd
from pythia_datasets import DATASETS

But I cannot find the pythia_datasets module:

import pandas as pd

from pythia_datasets import DATASETS
Traceback (most recent call last):

Cell In[3], line 1
from pythia_datasets import DATASETS

ModuleNotFoundError: No module named 'pythia_datasets'

The package should be available:
(pythia-book-dev) ejlevine@callisto:~$ conda list pythia

# packages in environment at /home/ejlevine/miniconda3/envs/pythia-book-dev:
#
# Name                    Version                   Build  Channel
pythia-datasets           2021.9.21          pyhd8ed1ab_0    conda-forge

And I can see the pythia_datasets directory in my ~/miniconda3/envs/pythia-book-dev/lib/python3.10/site-packages/pythia_datasets directory, with top-level files dated Sep 22 2021.

Can someone please help me fix this?

Thanks,

Eric Levine

Answer 1 · 2023-08-24T19:24:22.000Z

Hi @ejlevine, I wasn't able to recreate the failure. I ran that notebook in a freshly made conda environment on a Mac M1 laptop and everything worked as expected. So I'm not sure what's going on.

@ProjectPythia/infrastructure any ideas what to look at here?

Answer 2 · 2023-08-24T19:43:19.000Z

I don't know if this is related, but I noticed that we have some obsolete version pins in our environment file. I just opened #423 to remove those.

It might be worth trying to create a fresh version of the pythia-book-dev environment once #423 is merged.

Answer 3 · 2023-08-25T06:59:30.000Z

Hi @ejlevine , you are using the Spyder IDE to run the notebooks I believe.

The error is probably due to the IDE configuration, i.e. even though you ran conda activate pythia-book-dev in terminal, it only applies to within that terminal, and your Spyder IDE might probably be set to use a different conda env as Python interpreter; you can check this from its configs. When running notebooks via Spyder, it is probably using the conda env that is set as Python interpreter within its preferences.

For simplicity, could you please first try using jupyter lab to run pandas.ipynb as described here and let us know if you are able to access pythia-datasets?

If yes, you can either keep using jupyter-lab, which is recommended, or set your Spyder to use the pythia-book-dev environment for these notebooks.

Answer 4 · 2023-08-30T16:39:45.000Z

I found that the commands from pythia_datasets import DATASETS filepath = DATASETS.fetch ('enso_data.csv') work fine with ~/miniconda3/envs/pythia-book-dev/bin/ipython. I ran 'spyder --debug-info verbose' and got the following message: An error occurred while starting the kernel The Python environment or installation whose interpreter is located at /XXXXXX/miniconda3/envs/pythia-book-dev/bin/python doesn't have the spyder-kernels module or the right version of it installed (>= 2.4.0 and < 2.5.0). Without this module is [sic] not possible for Spyder to create a console for you. You can install it by activating your environment first (if necessary) and then running in a system terminal: conda install spyder-kernels=2.4 So I ran the following commands: conda config --set solver libmamba # The install didn't work with the 'classic' solver conda install spyder-kernels conda list | grep spyder spyder-kernels 2.4.4 py311h06a4308_0 And now the DATASETS commands work in spyder: import pandas as pd from pythia_datasets import DATASETS filepath = DATASETS.fetch ('enso_data.csv') filepathOut[4]: ~/.cache/pythia-datasets/enso_data.csv' thanks,

…

-eric

------- Original Message -------

On Friday, August 25th, 2023 at 12:59 AM, Orhan Eroglu ***@***.***> wrote: Hi ***@***.***(https://github.com/ejlevine) , you are using the Spyder IDE to run the notebooks I believe. The error is probably due to the IDE configuration, i.e. even though you ran conda activate pythia-book-dev in terminal, it only applies to within that terminal, and your Spyder IDE might probably be set to use a different conda env as Python interpreter; you can check this from its configs. When running notebooks via Spyder, it is probably using the conda env that is set as Python interpreter within its preferences. For simplicity, could you please first try using jupyter lab to run pandas.ipynb as described [here](https://foundations.projectpythia.org/preamble/how-to-use.html#:~:text=Now%20that%20your,~%0Ajupyter%20lab) and let us know if you are able to access pythia-datasets? If yes, you can either keep using jupyter-lab, which is recommended, or set your Spyder to use the pythia-book-dev environment for these notebooks. — Reply to this email directly, [view it on GitHub](#422 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AHWO57WFE5E4RAUVTWPGEOTXXBEN5ANCNFSM6AAAAAA35GABSI). You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 5 · 2023-08-30T16:52:38.000Z

@ejlevine thanks for sharing your solution with us!

Answer 6 · 2023-08-30T17:18:59.000Z

Is this a general-enough problem that we should include documentation on "how to run Foundations examples in Spyder?"

Answer 7 · 2023-08-30T17:36:04.000Z

Is this a general-enough problem that we should include documentation on "how to run Foundations examples in Spyder?"

This seems like an IDE-specific issue with conda environments, and I don't believe it should be documented under Pythia