JuliaMath/IntelVectorMath.jl

Cannot load library on Julia 1.3rc5: MethodError: no method matching iterate(::Nothing)

chriselrod opened this issue · 9 comments

chriselrod commented

On Julia 1.3rc5:

julia> using VML
[ Info: Precompiling VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc]
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:600
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:600
  iterate(::ExponentialBackOff) at error.jl:218
  ...
Stacktrace:
 [1] isempty(::Nothing) at ./essentials.jl:736
 [2] __init__() at /home/chriselrod/.julia/packages/VML/lA63S/src/setup.jl:5
 [3] top-level scope at /home/chriselrod/.julia/packages/VML/lA63S/src/setup.jl:18
 [4] include at ./boot.jl:328 [inlined]
 [5] include_relative(::Module, ::String) at ./loading.jl:1105
 [6] include at ./Base.jl:31 [inlined]
 [7] include(::String) at /home/chriselrod/.julia/packages/VML/lA63S/src/VML.jl:3
 [8] top-level scope at /home/chriselrod/.julia/packages/VML/lA63S/src/VML.jl:10
 [9] include at ./boot.jl:328 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1105
 [11] include(::Module, ::String) at ./Base.jl:31
 [12] top-level scope at none:2
 [13] eval at ./boot.jl:330 [inlined]
 [14] eval(::Expr) at ./client.jl:425
 [15] top-level scope at ./none:3
in expression starting at /home/chriselrod/.julia/packages/VML/lA63S/src/setup.jl:18
in expression starting at /home/chriselrod/.julia/packages/VML/lA63S/src/VML.jl:10
ERROR: Failed to precompile VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc] to /home/chriselrod/.julia/compiled/v1.3/VML/WlHoi_R5WJb.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917
aminya commented

What OS are you using? Have you installed MKL?

chriselrod commented

Oops -- I saw a weird error not mentioning libraries, and didn't investigate further in my rush to write a post.
I do not have MKL.jl installed, having built with MKL from source

julia> using LinearAlgebra

julia> BLAS.vendor()
:mkl

It looks like I should be getting a different error.
The README claims that

Alternatively you can install MKL directly from intel.

but from the setup script, it looks like this is not actually supported?

aminya commented

You need to either download MKL and put it on the path, or use the MKL.jl. Both work.

chriselrod commented 
> echo $LD_LIBRARY_PATH | grep linux/mkl/lib/intel64
/home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64_lin /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64_lin /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/mpi/intel64/libfabric/lib /opt/intel/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib/release /opt/intel/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib /opt/intel/compilers_and_libraries_2019.4.243/linux/ipp/lib/intel64 /opt/intel/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7 /opt/intel/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7 /opt/intel/compilers_and_libraries_2019.4.243/linux/daal/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/daal/../tbb/lib/intel64_lin/gcc4.4

Notice the third path listed. Looking inside:

> ls /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin
libmkl_avx2.so*        libmkl_avx.so*         libmkl_core.a    libmkl_gf_ilp64.a    libmkl_gf_lp64.so*     libmkl_intel_ilp64.a    libmkl_intel_lp64.so*    libmkl_lapack95_ilp64.a  libmkl_mc.so*        libmkl_sequential.so*  libmkl_vml_avx2.so*        libmkl_vml_avx.so*   libmkl_vml_mc2.so*  locale/
libmkl_avx512_mic.so*  libmkl_blas95_ilp64.a  libmkl_core.so*  libmkl_gf_ilp64.so*  libmkl_gnu_thread.a    libmkl_intel_ilp64.so*  libmkl_intel_thread.a    libmkl_lapack95_lp64.a   libmkl_rt.so*        libmkl_tbb_thread.a    libmkl_vml_avx512_mic.so*  libmkl_vml_cmpt.so*  libmkl_vml_mc3.so*
libmkl_avx512.so*      libmkl_blas95_lp64.a   libmkl_def.so*   libmkl_gf_lp64.a     libmkl_gnu_thread.so*  libmkl_intel_lp64.a     libmkl_intel_thread.so*  libmkl_mc3.so*           libmkl_sequential.a  libmkl_tbb_thread.so*  libmkl_vml_avx512.so*      libmkl_vml_def.so*   libmkl_vml_mc.so*

Or, to be more succinct:

> ls /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin | grep vml
libmkl_vml_avx2.so
libmkl_vml_avx512_mic.so
libmkl_vml_avx512.so
libmkl_vml_avx.so
libmkl_vml_cmpt.so
libmkl_vml_def.so
libmkl_vml_mc2.so
libmkl_vml_mc3.so
libmkl_vml_mc.so

I already have VML on my load path, so I don't think I should have to install anything else?

  • 👀1
aminya commented

@Crown421 has a Linux system. So he can help you better.

Crown421 commented

I had the same error on one of my systems (iterate over Nothing), I thought I had fixed it with 0094836

On a fresh julia REPL, can you run and see if it gives you something back?

julia> using Libdl
julia> Libdl.dlpath(:libmkl_rt)

Also, can you check your local setup.jl file / update the package? Because with the aforementioned commit there should be nothing to iterate over in the __init__.

chriselrod commented 
julia> using Libdl

julia> Libdl.dlpath(:libmkl_rt)
"/home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin/libmkl_rt.so"

I am currently on VML master. I still get the same error.

julia> using VML
[ Info: Precompiling VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc]
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:602
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:602
  iterate(::ExponentialBackOff) at error.jl:253
  ...
Stacktrace:
 [1] isempty(::Nothing) at ./essentials.jl:738
 [2] __init__() at /home/chriselrod/.julia/packages/VML/SaEFf/src/setup.jl:5
 [3] top-level scope at /home/chriselrod/.julia/packages/VML/SaEFf/src/setup.jl:18
 [4] include(::Module, ::String) at ./Base.jl:377
 [5] include(::String) at /home/chriselrod/.julia/packages/VML/SaEFf/src/VML.jl:3
 [6] top-level scope at /home/chriselrod/.julia/packages/VML/SaEFf/src/VML.jl:10
 [7] include(::Module, ::String) at ./Base.jl:377
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:449
 [11] top-level scope at ./none:3
in expression starting at /home/chriselrod/.julia/packages/VML/SaEFf/src/setup.jl:18
in expression starting at /home/chriselrod/.julia/packages/VML/SaEFf/src/VML.jl:10
ERROR: Failed to precompile VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc] to /home/chriselrod/.julia/compiled/v1.4/VML/WlHoi_8nmxv.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1263
 [3] _require(::Base.PkgId) at ./loading.jl:1022
 [4] require(::Base.PkgId) at ./loading.jl:920
 [5] require(::Module, ::Symbol) at ./loading.jl:915

Checking the contents of SaEFf/src/setup.jl

function __init__()
    MKLpkgid = Base.PkgId(Base.UUID("33e6dc65-8f57-5167-99aa-e5a354878fb2"), "MKL")
    mklpath = Base.locate_package(MKLpkgid)
    if !isempty(mklpath)
        libpath = normpath(joinpath(dirname(mklpath), "../deps/usr/lib"))
        push!(Libdl.DL_LOAD_PATH, libpath)
    elseif isempty(Libdl.find_library(rtlib))
        error("Could not find MKL shared libraries. Please add MKL.jl or install MKL via the intel website. See the github repository for more details.)")
    end

    Libdl.dlopen(rtlib, RTLD_GLOBAL)
    Libdl.dlopen(corelib, RTLD_GLOBAL) # maybe only needed on mac
    Libdl.dlopen(lib, RTLD_GLOBAL)

end

The error comes from:

julia> isempty(nothing)
ERROR: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:602
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:602
  iterate(::ExponentialBackOff) at error.jl:253
  ...
Stacktrace:
 [1] isempty(::Nothing) at ./essentials.jl:738
 [2] top-level scope at REPL[31]:1

This happens because I do not have MKL.jl installed:

julia> MKLpkgid = Base.PkgId(Base.UUID("33e6dc65-8f57-5167-99aa-e5a354878fb2"), "MKL")
MKL [33e6dc65-8f57-5167-99aa-e5a354878fb2]

julia> mklpath = Base.locate_package(MKLpkgid)

julia> mklpath === nothing
true

I think what you want to do is a check vs nothing, not a isempty.

Crown421 commented

Very interesting, I thought the machine I tested this on also does not have MKL.jl, but instead uses MKL_slim from the AUR.
Well, testing on WSL Ubuntu confirms that you are right, I just changed it.
Writing something for Travis to check this against apt installed MKL is on my list for the weekend.

chriselrod commented

Very interesting, I thought the machine I tested this on also does not have MKL.jl, but instead uses MKL_slim from the AUR.
Well, testing on WSL Ubuntu confirms that you are right, I just changed it.
Writing something for Travis to check this against apt installed MKL is on my list for the weekend.

Thanks, I just tested and confirmed that it now works locally!