alan-turing-institute/PDSampler.jl

Julia 0.6.3 & 0.7-alpha compat

tlienart opened this issue · 4 comments

tlienart commented

Julia 0.6.3 has just been released

PDSampler still passes tests as expected. Though when testing I had to

Pkg.build("FFTW")
Pkg.build("Distributions")
Pkg.build("PDMats")
Pkg.build("PDSampler")

This had likely nothing to do with 0.6.3 but rather with the fact that some packages had been updated and needed to be re-built.

Julia 0.7-alpha has just been released

Consequently this package will be updated (slowly). Progress updated here.

using Pkg
Pkg.add("PDSampler")
Pkg.test("PDSampler")

unsurprisingly throws quite a few errors:

   Testing PDSampler
ERROR: LoadError: ArgumentError: Module DiffResults not found in current path.
Run `Pkg.add("DiffResults")` to install the DiffResults package.
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:868
 [2] include at ./boot.jl:314 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1071
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope
 [6] eval at ./boot.jl:316 [inlined]
 [7] eval(::Expr) at ./client.jl:394
 [8] macro expansion at ./none:3 [inlined]
 [9] top-level scope at ./<missing>:0
in expression starting at /Users/tlienart/.julia/packages/Klara/iHrv/src/Klara.jl:5
ERROR: LoadError: Failed to precompile Klara to /Users/tlienart/.julia/compiled/v0.7/Klara/LH3t.ji.
Stacktrace:
 [1] error at ./error.jl:33 [inlined]
 [2] compilecache(::Base.PkgId) at ./loading.jl:1207
 [3] _require(::Base.PkgId) at ./loading.jl:978
 [4] require(::Base.PkgId) at ./loading.jl:878
 [5] require(::Module, ::Symbol) at ./loading.jl:873
 [6] include at ./boot.jl:314 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1071
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] top-level scope
 [10] eval at ./boot.jl:316 [inlined]
 [11] eval(::Expr) at ./client.jl:394
 [12] macro expansion at ./none:3 [inlined]
 [13] top-level scope at ./<missing>:0
in expression starting at /Users/tlienart/.julia/packages/PDSampler/qauV/src/PDSampler.jl:6
ERROR: LoadError: Failed to precompile PDSampler to /Users/tlienart/.julia/compiled/v0.7/PDSampler/nwMk.ji.
Stacktrace:
 [1] error at ./error.jl:33 [inlined]
 [2] compilecache(::Base.PkgId) at ./loading.jl:1207
 [3] _require(::Base.PkgId) at ./loading.jl:1007
 [4] require(::Base.PkgId) at ./loading.jl:878
 [5] require(::Module, ::Symbol) at ./loading.jl:873
 [6] include at ./boot.jl:314 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1071
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] include(::String) at ./client.jl:393
 [10] top-level scope
in expression starting at /Users/tlienart/.julia/packages/PDSampler/qauV/test/runtests.jl:1
ERROR: Package PDSampler errored during testing
Stacktrace:
 [1] #test#57(::Bool, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/Types.jl:359
 [2] #test at ./<missing>:0 [inlined]
 [3] #test#35(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:216
 [4] test at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:205 [inlined]
 [5] #test#34 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:202 [inlined]
 [6] test at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:202 [inlined]
 [7] #test#33 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:201 [inlined]
 [8] test at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:201 [inlined]
 [9] #test#32 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:200 [inlined]
 [10] test(::String) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:200
 [11] top-level scope
  • I don't know why Klara is still in there somewhere given that I thought I had explicitly removed all references to it
  • DiffResults will likely need to be added to REQUIRE
tlienart commented

Progressing...

using Pkg
Pkg.add("DiffResults")
Pkg.build("DiffResults") # needed
using PDSampler

ok so that didn't help

[ Info: Precompiling module PDSampler
ERROR: LoadError: ArgumentError: Module DiffResults not found in current path.
Run `Pkg.add("DiffResults")` to install the DiffResults package.
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:868
 [2] include at ./boot.jl:314 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1071
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope
 [6] eval at ./boot.jl:316 [inlined]
 [7] eval(::Expr) at ./client.jl:394
 [8] macro expansion at ./none:3 [inlined]
 [9] top-level scope at ./<missing>:0
in expression starting at /Users/tlienart/.julia/packages/Klara/iHrv/src/Klara.jl:5
tlienart commented

Trying to run PDSampler.jl file bit by bit to see what works and what does not.

  • well to start with, it looks like ApproxFun doesn't work in Julia 0.7 (as far as I can tell). So that's a starting point that needs to be fixed...
    • a ton of depr warnings
    • says AbstractFFTs not found (definitely is because using AbstractFFTs works just fine)
  • Pkg.add("StatsBase") followed by using StatsBase: autocov seems to work fine
  • same for Polynomials, DataStructures, ProgressMeter`
  • Distributions causes issues.
    • a ton of depr warnings
    • UndefVarError: RangeGeneratorInt not defined --> hmm, I don't know what that is

here's the script contained in PDSampler.jl, the lines marked with XXX indicate something wrong upstream (not this package directly). TODO indicates a deprecation warning that should be fixed.

# XXX # using ApproxFun
using StatsBase: autocov
using Polynomials:
        Poly,
        roots,
        polyint,
        polyval
using DataStructures:
        PriorityQueue,
        enqueue!,
        dequeue!,
        peek
using ProgressMeter
# XXX # using Distributions: Beta

include("models/mvgaussian.jl")
include("models/tmvgaussian.jl")
# XXX # include("models/logreg.jl")
include("models/pmf.jl")

include("geometry.jl")
include("ippsampler.jl")
# TODO # include("path.jl")
# XXX (Beta) # include("kernels.jl")
# TODO # include("simulate.jl")

# TODO # include("local/event.jl")
# TODO # include("local/factorgraph.jl")
# TODO # include("local/simulate.jl")

The deprecation warnings should be fixable fairly easily. For the errors we'll have to wait a bit that the other packages get updated. Apart from one error that may be fixable

include("models/logreg.jl")
ERROR: LoadError: syntax: local variable #s1 cannot be used in closure declaration
Stacktrace:
 [1] include at ./boot.jl:314 [inlined]
 [2] include_relative(::Module, ::String) at ./loading.jl:1071
 [3] include(::Module, ::String) at ./sysimg.jl:29
 [4] include(::String) at ./client.jl:393
 [5] top-level scope
in expression starting at /Users/tlienart/.julia/v0.6/PDSampler/src/models/logreg.jl:51

Well, it turns out that last error about closure is triggered by

function gradloglik_cv(lr::LogReg, wstar::Vector{<:Real})
    # Unbiased estimate of ∇U_j(x) using control variates
    gll_star = gradloglik(lr, wstar)

    function gll_cv(w::Vector{<:Real}, i=rand(1:lr.n))
        tw, tws = glli(w, lr, i), glli(wstar, lr, i)
        # unbiased estimate of the gradient
        gll_star + lr.n * lr.y[i] * (tw * lr.X[i, :] - tws * lr.X[i,:])
    end
    return gll_cv
end

this seems to be a bug (JuliaLang/julia#27268) so should probably just wait at this point.

tlienart commented

re warnings

  • Base.Range is deprecated, use AbstractRange instead this is breaking 0.6-0.7 (AbstractRange is not available in 0.6) so waiting for now
  • Deprecated syntax immutable use struct --> all done. (17e3108)

for all the rest

  1. wait until ApproxFun, Distributions have been updated
  2. wait until the closure bug is fixed.
tlienart commented

#34