`Pkg.test` should maybe instantiate the full test project
Keno opened this issue · 2 comments
Consider the following
$ find -type f -exec batcat --style header {} +
File: ./test/runtests.jl
using SciMLSensitivity
File: ./docs/Project.toml
[deps]
SciMLSensitivity = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
[sources]
SciMLSensitivity = { url = "https://github.com/Keno/SciMLSensitivity.jl", rev = "kf/mindep" }
File: ./Project.toml
name = "PkgTestWeirdness"
uuid = "78ed732d-4f3c-41ca-8ab7-3ddcca345aff"
[workspace]
projects = ["docs"]
[extras]
SciMLSensitivity = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
[targets]
test = ["SciMLSensitivity"]
Then at the REPL
(PkgTestWeirdness) pkg> resolve
shell> rm -rf ~/.julia/packages/SciMLSensitivity/ ; simulate moving to CI
(PkgTestWeirdness) pkg> test
Testing PkgTestWeirdness
ERROR: expected package `SciMLSensitivity [1ed8b502]` to exist at path `/home/keno/.julia/packages/SciMLSensitivity/IpVTR`
Ordinary instantiate (as both CI and Pkg.test
run) does not fix this:
(PkgTestWeirdness) pkg> instantiate
0 dependencies successfully precompiled in 0 seconds
1 dependency errored.
For a report of the errors see `julia> err`. To retry use `pkg> precompile`
(PkgTestWeirdness) pkg> test
Testing PkgTestWeirdness
ERROR: expected package `SciMLSensitivity [1ed8b502]` to exist at path `/home/keno/.julia/packages/SciMLSensitivity/IpVTR`
instantiate --workspace
does fix this, but it's not what CI runs (though it could).
Possible solutions I can see:
-
The Pkg.instantiate that
Pkg.test
does should instantiate the full test project, not just the toplevel project. The issues here is that this breaks the case whereSciMLSensitivity
is not already in the manifest. -
The
resolve
thatPkg.test
does should maybe ignore the fact that the manifest is non-instantiated. The pkg is pinned and the toplevel manifest should have sufficient information for it to do everything that it needs. Then, we could instantiate after the resolve, which would avoid the previously mentioned problem.
Probably has the same root cause as #860 (hello issue from 6 years ago). Also #3859. Basically, resolve
errors if a package added via URL is not downloaded. Can be reprod easier via:
(TestInst) pkg> st
Project TestInst v0.1.0
Status `~/JuliaTests/TestInst/Project.toml`
→ [1ed8b502] SciMLSensitivity v7.56.2 `https://github.com/Keno/SciMLSensitivity.jl#kf/mindep`
Info Packages marked with → are not downloaded, use `instantiate` to download
(TestInst) pkg> resolve
ERROR: expected package `SciMLSensitivity [1ed8b502]` to exist at path `/Users/kristoffercarlsson/.julia/packages/SciMLSensitivity/IpVTR`
Your PR fixes two of the tree ways I tried, but this not the third one. Reproducer:
$ find -type f -exec batcat --style header {} +
File: ./test/runtests.jl
using SciMLSensitivity
File: ./test/Project.toml
[deps]
SciMLSensitivity = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
File: ./docs/Project.toml
[deps]
SciMLSensitivity = "1ed8b502-d754-442c-8d5d-10ac956f44a1"
[sources]
SciMLSensitivity = { url = "https://github.com/Keno/SciMLSensitivity.jl", rev = "kf/mindep" }
File: ./Project.toml
name = "PkgTestWeirdness"
uuid = "78ed732d-4f3c-41ca-8ab7-3ddcca345aff"
[workspace]
projects = ["docs", "test"]
File: ./src/PkgTestWeirdness.jl
module PkgTestWeirdness
end
Same setup script and then:
(PkgTestWeirdness) pkg> test
[snip]
Testing Running tests...
ERROR: LoadError: ArgumentError: Package SciMLSensitivity [1ed8b502-d754-442c-8d5d-10ac956f44a1] is required but does not seem to be installed:
- Run `Pkg.instantiate()` to install all recorded dependencies.