error in Calphy: NameError: name 'Calculation' is not defined

Question

error in Calphy: NameError: name 'Calculation' is not defined

lfzhu-phys opened this issue a year ago · 24 comments

I have a problem to run Calphy with both MEAM potential and MTP potential.
@srmnitc Could you please have a look and give me some suggestions? Thanks a lot.

The error is like following:

/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pandas/core/computation/expressions.py:21: UserWarning: Pandas requires version '2.8.0' or newer of 'numexpr' (version '2.7.3' currently installed).
  from pandas.core.computation.check import NUMEXPR_INSTALLED
Traceback (most recent call last):
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_base/cli/__main__.py", line 3, in <module>
    main()
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_base/cli/control.py", line 59, in main
    args.cli(args)
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_base/cli/wrapper.py", line 37, in main
    job_wrapper_function(
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_base/jobs/job/wrapper.py", line 184, in job_wrapper_function
    job.run()
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_base/jobs/job/wrapper.py", line 129, in run
    self.job.run_static()
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_atomistics/calphy/job.py", line 673, in run_static
    calc = self._create_calc()
  File "/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pyiron_atomistics/calphy/job.py", line 469, in _create_calc
    calc = Calculation()
NameError: name 'Calculation' is not defined

Answer 1 · 2023-10-26T08:07:34.000Z

Looks to my like numexpr must be updated to version >=2.8.0. Updating packages on cmti is a bit problematic RN. @niklassiemer can you say something about this?

Answer 2 · 2023-10-26T08:18:51.000Z

The warning 'from pandas.core.computation.check import NUMEXPR_INSTALLED' keeps popping out for every notebook. But it seems that it does not influence other calculations.

Answer 3 · 2023-10-26T08:26:55.000Z

It depends on the kernel you are using. The pyiron/latest kernel should be updatable :) I assume you are using that one since you use calphy? I do not know what exactly is going wrong, though.

Answer 4 · 2023-10-26T08:27:40.000Z

How to update the kernel? Sorry:)

Answer 5 · 2023-10-26T08:29:55.000Z

In the jupyterhub you should have a new python kernel available named pyiron_latest. At least for non-MTPs this version should work.

Answer 6 · 2023-10-26T08:31:23.000Z

Ah, you do have to take care of #1195

Answer 7 · 2023-10-26T08:34:34.000Z

Thanks a lot. I changed the kernel to the latest one. But the error remains......

Answer 8 · 2023-10-26T08:46:46.000Z

@pmrv Thanks a lot.

The following is my setup:

al_sol.calc_free_energy(temperature=500,
pressure=0,
reference_phase="solid",
n_equilibration_steps=5000,
n_switching_steps=5000)

Do you mean I should add "al_sol.input.npt = True" following the code? But it also doesn't work...

Ah, you do have to take care of #1195

Answer 9 · 2023-10-26T10:01:38.000Z

@lfzhu-phys I can reproduce this error on the cluster, I am checking now what is wrong..

Answer 10 · 2023-10-26T10:10:50.000Z

@lfzhu-phys @niklassiemer On the default cluster environment, there is no calphy installed, which is why the run fails. However, when I follow @niklassiemer 's instruction and change kernel on the top right to pyiron/latest (Python 3.10), everything works for me and the calculations runs. This is the input setup I used:

from pyiron_atomistics import Project
pr = Project('test')
struct = pr.create.structure.ase.bulk("Al").repeat(4)
job = pr.create.job.Calphy("a")
job.structure = struct
job.potential = '2005--Mendelev-M-I--Al-Fe--LAMMPS--ipr1'
job.input.npt = True
job.calc_free_energy(temperature=500,
    pressure=0,
    reference_phase="solid",
    n_equilibration_steps=5000,
    n_switching_steps=5000)
job.run()

Answer 11 · 2023-10-26T10:36:15.000Z

I also checked, when using pyiron/latest we get

>>> import numexpr
>>> numexpr.__version__
'2.8.7'

So this should be fine. @lfzhu-phys , do you know how to change the kernel?

Answer 12 · 2023-10-26T10:44:02.000Z

@mbruns91 Thanks. I already changed the kernel and the warning is gone now:)

Answer 13 · 2023-10-26T10:46:05.000Z

@lfzhu-phys @niklassiemer On the default cluster environment, there is no calphy installed, which is why the run fails. However, when I follow @niklassiemer 's instruction and change kernel on the top right to pyiron/latest (Python 3.10), everything works for me and the calculations runs. This is the input setup I used:
from pyiron_atomistics import Project
pr = Project('test')
struct = pr.create.structure.ase.bulk("Al").repeat(4)
job = pr.create.job.Calphy("a")
job.structure = struct
job.potential = '2005--Mendelev-M-I--Al-Fe--LAMMPS--ipr1'
job.input.npt = True
job.calc_free_energy(temperature=500,
    pressure=0,
    reference_phase="solid",
    n_equilibration_steps=5000,
    n_switching_steps=5000)
job.run()

@srmnitc Thank you very much. Could you please try to run it with 2 cores parallel? I still can reproduce the same error. But, if I run it on the server without submitting it to the cluster, it starts to run and generate outputs. At the end it ends up with an error like this

FileNotFoundError: /cmmc/u/lfzhu/TOR-TILD/TOR-TILD_Al/PBE/Calphy/Al_test/a_hdf5/a/conf.equilibration.dump file not found

Answer 14 · 2023-10-26T10:53:43.000Z

There are two issues here, so far clarity I will split them up:

Changing kernel works on login node (at least to start the run) but do not work if submitted to queue. Is this correct @lfzhu-phys ? If that is the case, somehow the kernel is not selected correctly on the nodes? @niklassiemer @mbruns91 would need to take a look...Idk if the last few steps here are related.
The calculation starts, but fails with error FileNotFoundError: /cmmc/u/lfzhu/TOR-TILD/TOR-TILD_Al/PBE/Calphy/Al_test/a_hdf5/a/conf.equilibration.dump file not found. This is a known issue, and I have released a fix. If the calphy version on cluster is updated, this would be fixed (I have emailed @niklassiemer already). But on the brighter side, this means that the calculation worked. @lfzhu-phys you can go into the job folder, and you will see a file report.yaml with your results.

Answer 15 · 2023-10-26T10:57:39.000Z

@srmnitc Yes, both issues are exactly as you stated.

I also found the report.yaml file in the folder.

Thank you very much.

Answer 16 · 2023-10-26T11:44:32.000Z

Ok, for point 1. we maybe should wait for @niklassiemer , as I'm also not familiar with all the details of the new env yet. I guess it's something with modules not being loaded or unloaded correctly when logging in.

However, regarding point 2, under kernel pyiron_latest, the newest calphy version is available now!

Answer 17 · 2023-10-26T12:46:40.000Z

On submitting to queue, it should as far as I understood load the same env variables as on the login node.... I mean, it is not possible to modify the bashrc differently for different jobs, since you do not know when they will start... you could try to delete the module load pyiron/dev from your bashrc and only load that module if you want to use python from the command line, since this could maybe change environment variables?

Answer 18 · 2023-10-26T12:48:32.000Z

There also is a pyiron/latest module (albeit hidden right now) which, however, would then probably confuse calculations in the pyiron/dev environment?

Answer 19 · 2023-10-27T08:31:07.000Z

Dear all, thanks a lot for all your quick reply.

Now, the point 1 is fixed and I can run Calphy jobs on the login node.

However, point 2 still remains. I comment out "module load pyiron/dev" in bashrc file. Then the job complains that it can not find pyiron_base. Kindly let me know if there are more suggestions.

Answer 20 · 2023-10-27T08:51:34.000Z

@lfzhu-phys Does this mean everything is working now?

Answer 21 · 2023-10-27T08:51:59.000Z

As a reference for others, @srmnitc wrote a nice summary how to setup the new environment on https://gitlab.mpcdf.mpg.de/pyiron/faqs/-/issues/2 .

Answer 22 · 2023-10-27T08:55:30.000Z

@lfzhu-phys Does this mean everything is working now?

Point 2 is fixed, but point 1 is still there. If I comment out "module load pyiron/dev" in bashrc file, the job complains that it can not find pyiron_base. If I don't comment out, the old issue remains "NameError: name 'Calculation' is not defined".

Answer 23 · 2023-10-27T09:16:22.000Z

@lfzhu-phys I presume the following lines are missing in the new kernel:

"env": {
  "PATH": "/cmmc/ptmp/<username>/<envname>/bin:${PATH}",
  "CONDA_PREFIX": "/cmmc/ptmp/<username>/<envname>"
}

@niklassiemer Can you check if the lines are available in the latest kernel? Or add them if they are not?

Answer 24 · 2023-11-14T09:50:02.000Z

Both issues are fixed now. Thanks you all very much:)