jarvist/Quante.jl

@distributed does not work for multi-core 1 and 2 electron integrals

Closed this issue · 4 comments

jarvist commented

The new @distributed parallel macro does not seem to actually work when running on multiple cores.
The error appears that it may just be a matter of understanding the new Package manager sufficiently so that the correct environment is instantiated on each process.

[Fri28Sep-16:45]jarvist@nmscde005688:~/REPOS/Quante.jl/test/ 
> julia-1.0.0 -p auto runtests.jl 
ERROR: LoadError: On worker 2:
ArgumentError: Package Quante [top-level] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

_require at ./loading.jl:923
require at ./loading.jl:852
#2 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/Distributed.jl:77
#116 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:276
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:65
#102 at ./task.jl:259

It may just be a matter of installing the package properly via the package manager to begin with!

mszep commented

Figured out a workaround (or is it a fix, I don't know):

  • In the Quante.jl root directory, open a julia prompt and hit ] to enter the package manager
  • type add . to install the curent package
  • now invoking runtests.jl with -p auto should work!
mszep commented

There's probably a better workflow out there for development, currently reading through https://docs.julialang.org/en/v1/stdlib/Pkg/index.html for more info...

jarvist commented

OK, now that the Julia 1.0 package is registered, you can install it directly. And once you do that, the tests run fine under @distributed on multiple cores.

I wish I understood more about how to set up development for the new package manager, but for users I think this should be fine.

[Tue02Oct-20:33]jarvist@Asriel:~/.julia/packages/Quante/test/ 
> julia-1.0.0 -p 2 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.0 (2018-08-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("runtests.jl")
[ Info: Precompiling Quante [62e6779b-560f-53ed-829f-06ad43eb5710]

...

Test Summary: | Pass  Total
Quante.jl     |  157    157
All tests passed! 8^)
julia> 
      From worker 3:	.............
      From worker 2:	...........
  • 🎉1
jarvist commented

I tested this a little more, and it seems fine. The Julia 1.0 @distributed macro requires a correctly registered (at least locally) package, you can't just add ../src/ into the path etc.