h5py problem
Closed this issue · 4 comments
Hi,
I am trying to solve 3D within spins-b/invdes example of wdm2.py on Ubuntu.
However it gives H5PY error, which I tried many things to overcome, but couldn't and finally writing here in need of an answer. I started Maxwell servers with the ./start_maxwell example and their logs are just fine as till I give "python3 wdm2.py" command from another terminal:
simserver.log:
[2020-05-13 03:54:42,057][INFO][simserver][main] Solver directory set to /home/iclibera/Downloads/maxwell-b-master/maxwell-server/maxwell-solver/
[2020-05-13 03:54:42,091][INFO][simserver][main] Number of GPUs detected on system: 1
[2020-05-13 03:54:42,091][INFO][simserver][main] Number of GPUs used per solve: 1
[2020-05-13 03:54:42,091][INFO][simserver][main] Ready to accept simulations.
[2020-05-13 03:55:21,642][INFO][simserver][main] Solving 127.0.0.1:maxwell-20200513-035519-6ad799e8-94b4-11ea-9d77-1831bf1f80db as simulation 0
[2020-05-13 03:55:24,678][INFO][simserver][main] Simulation 0 ended with code 1
[2020-05-13 03:55:25,195][INFO][simserver][main] Solving 127.0.0.1:maxwell-20200513-035519-6ad9366e-94b4-11ea-8cf1-ab50cbd49ffb as simulation 0
[2020-05-13 03:55:28,230][INFO][simserver][main] Simulation 0 ended with code 1
webserver.py:
Serving at ('', 9041)
the output of the other terminal is as follows:
user@user:~/Downloads/spins-b/examples/invdes/wdm2$ python3 wdm2.py
[2020-05-13 03:55:04,036][INFO][solver][run_plan] Setting up workspace.
[2020-05-13 03:55:04,036][INFO][solver][run_plan] Running transformation opt_cont0.
/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/scipy/sparse/data.py:74: ComplexWarning: Casting complex values to real discards the imaginary part
self._deduped_data().astype(dtype, casting=casting, copy=copy),
/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/scipy/sparse/data.py:74: ComplexWarning: Casting complex values to real discards the imaginary part
self._deduped_data().astype(dtype, casting=casting, copy=copy),
/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/scipy/sparse/data.py:74: ComplexWarning: Casting complex values to real discards the imaginary part
self._deduped_data().astype(dtype, casting=casting, copy=copy),
../../../spins/fdfd_solvers/maxwell.py:378: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
with h5py.File(file_prefix + 'r') as f:
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/h5py/_hl/files.py", line 199, in make_fid
fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/h5py/_hl/files.py", line 202, in make_fid
fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "wdm2.py", line 300, in
main()
File "wdm2.py", line 63, in main
problem_graph.run_plan(plan, ".")
File "../../../spins/invdes/problem_graph/solver.py", line 61, in run_plan
work.run(transformation_param, event_data)
File "../../../spins/invdes/problem_graph/workspace.py", line 134, in run
transform(self.get_object(node.parametrization), event_data)
File "../../../spins/invdes/problem_graph/creator_opt.py", line 80, in call
self._start(param)
File "../../../spins/invdes/problem_graph/creator_opt.py", line 109, in _start
monitor_list=self.monitor_lists.start_monitors)
File "../../../spins/invdes/problem_graph/workspace.py", line 345, in write
[self._work.get_object(mon) for mon in monitor_list], param)
File "../../../spins/invdes/problem/graph_executor.py", line 87, in eval_fun
_eval_fun_vals(fun_vals, fun_map, graph, top_sorted_nodes, param)
File "../../../spins/invdes/problem/graph_executor.py", line 179, in eval_fun_vals
for node, node_val in zip(heavy_node_block, heavy_node_vals):
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
yield fs.pop().result()
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "../../../spins/invdes/problem/graph_executor.py", line 175, in
heavy_node_vals = executor.map(lambda args: _eval_fun(*args),
File "../../../spins/invdes/problem/graph_executor.py", line 122, in _eval_fun
return fun_map[node].eval(input_vals)
File "../../../spins/invdes/problem_graph/creator_em.py", line 309, in eval
return self._simulate(input_val[0])
File "../../../spins/invdes/problem_graph/creator_em.py", line 372, in _simulate
bloch_vec=self._bloch_vector,
File "../../../spins/fdfd_solvers/maxwell.py", line 378, in solve
with h5py.File(file_prefix + 'r') as f:
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/h5py/_hl/files.py", line 408, in init
swmr=swmr)
File "/home/linuxbrew/.linuxbrew/Cellar/python/3.7.7_1/lib/python3.7/site-packages/h5py/_hl/files.py", line 204, in make_fid
fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 108, in h5py.h5f.create
OSError: Unable to create file (unable to open file: name = '/tmp/tmp6i1f0xqj/maxwell-20200513-035519-6ad9366e-94b4-11ea-8cf1-ab50cbd49ffb.E_xr', errno = 17, error message = 'File exists', flags = 15, o_flags = c2)
Is there anyone can make a suggestion or at least an assumption on what may be causing this?
Thank you in advance,
Bera
I have traced the clues and solved the problem:
- at maxwell-b-virenv do this if you are also having the same errors:
- pip3 install h5py
- pip3 install mpi4py
- pip3 install wheel
I was probably having these errors because I did something wrong with maxwell-b installation. But, fixing with the above finally worked.
Thank you.
It's great that you were able to solve the problem. You probably figured this out, but I'll just point it out for the benefit of others:
The code that is output from Maxwell ("Simulation 0 ended with code 1") is the Unix exit code. Any non-zero code indicates an error. For Maxwell there is the server log, which was posted above and indicates which simulations have been launched. However, for every simulation, there is a specific log file (by default this is under /tmp/maxwell-server-files/), which should indicate why a specific simulation is failing.
Have you figured this problem? thx