Behavior of LinkingTo for source installations
Closed this issue · 3 comments
Reproducer, as reported by @eddelbuettel :
$ docker run --rm -it rocker/r2u:jammy bash -c "Rscript <(wget -O- https://raw.githubusercontent.com/eddelbuettel/demo-tldbsm/master/useRUniverse.R | sed 's/\"RcppInt64\", //')"
...
<snip>
...
Setting up r-cran-purrr (1.0.2-1.ca2204.1) ...
Setting up r-cran-arrow (15.0.1-1.ca2204.1) ...
Installing packages into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
also installing the dependency ‘RcppInt64’
trying URL 'https://cloud.r-project.org/src/contrib/RcppInt64_0.0.4.tar.gz'
Content type 'application/x-gzip' length 15227 bytes (14 KB)
==================================================
downloaded 14 KB
trying URL 'https://tldbsm.r-universe.dev/bin/linux/jammy/4.3/src/contrib/tiledb_0.25.0.106.tar.gz'
Content type 'application/x-gzip' length 29753910 bytes (28.4 MB)
==================================================
downloaded 28.4 MB
trying URL 'https://tldbsm.r-universe.dev/bin/linux/jammy/4.3/src/contrib/tiledbsoma_1.9.4.tar.gz'
Content type 'application/x-gzip' length 44328205 bytes (42.3 MB)
==================================================
downloaded 42.3 MB
* installing *source* package ‘RcppInt64’ ...
** package ‘RcppInt64’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
using C++ compiler: ‘g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I../inst/include -I'/usr/lib/R/site-library/Rcpp/include' -fpic -g -O2 -ffile-prefix-map=/build/r-base-14Q6vq/r-base-4.3.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I../inst/include -I'/usr/lib/R/site-library/Rcpp/include' -fpic -g -O2 -ffile-prefix-map=/build/r-base-14Q6vq/r-base-4.3.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c int64.cpp -o int64.o
g++ -std=gnu++17 -I"/usr/share/R/include" -DNDEBUG -I../inst/include -I'/usr/lib/R/site-library/Rcpp/include' -fpic -g -O2 -ffile-prefix-map=/build/r-base-14Q6vq/r-base-4.3.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c nanotime.cpp -o nanotime.o
g++ -std=gnu++17 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -o RcppInt64.so RcppExports.o int64.o nanotime.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-RcppInt64/00new/RcppInt64/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppInt64)
* installing *binary* package ‘tiledb’ ...
* DONE (tiledb)
* installing *binary* package ‘tiledbsoma’ ...
* DONE (tiledbsoma)
The downloaded source packages are in
‘/tmp/RtmpQT3PL5/downloaded_packages’
tiledbsoma: 1.9.4
tiledb-r: 0.25.0.106
tiledb core: 2.21.1
libtiledbsoma: 2.21.1
R: R version 4.3.3 (2024-02-29)
OS: Ubuntu 22.04.1 LTSwhich shows that RcppInt64, which is a LinkingTo-only dep for tiledb, is installed from source. In fact, to trigger this with this setup, we just need to install tiledb, not tiledbsoma.
The thing is that, in the version.check path (default), when looking for binary packages from system repos, LinkingTo is not queried for a simple reason: these are build-time deps, not run-time deps, so they are not needed for binary packages. Then binary versions are checked against source versions, and if source versions are available and preferred, install.packages proceeds as usual for those packages, which means that LinkingTo-only deps are installed from source too.
This is fine in general, because LinkingTo-only deps tend to be header-only, so it doesn't matter much if they are installed from source or not. The case above is particularly special and surprising due to two reasons:
- Even if RcppInt64 is header-only, it gets compiled because it provides a couple of compiled functions.
- r-universe repos provide binaries, and RcppInt64, from CRAN, is the only one that gets compiled.
So the question is, after the packages that should be installed from source are determined here, should we make a pass to try to install LinkingTo packages from the system repos?
Good analysis. Mostly nodding along. May mostly be a R issue, really, as it fails to differentiate between compile-time and run-time. But given that ... bspm may have to treat them as hard depends. Uggh.
When a CRAN-like repo provides a binary, LinkingTo-only packages are still pulled, which doesn't make any sense of course and it's an R issue as you say.
Here we can still avoid pulling them (if the system repo doesn't declare them as hard deps, of course), except for this case in which we resort to install.packages. Then an additional call to install_sys would at least pull RcppInt64 from the system repos. Not great, but better than the alternative, which is pulling them always (and I'm looking e.g. at BH, of course).
Yep long-time warts indeed, but we play the hand we are dealt.
For r-cran-bh we sometimes cheat and make it an empty virtual package depending in libboost-dev or alike.