Test suite failure on 32bit architectures
tillea opened this issue · 12 comments
Describe the bug
The Debian packaged version of future.apply started throwing test suite errors after upgrading from version 1.11.0 to 1.11.1 on 32bit architectures (armel, armhf, i386). For instance you can inspect the log for i386 which ends with
> stopifnot(all(sapply(y, FUN = identical, oMaxSize)))
>
> message("- approximately invariant to chunk size ...")
> - approximately invariant to chunk size ...
maxSize <- sizes[["FUN"]] + sizes[["X"]] / length(X)
> maxSize <- 3.0 * maxSize ## Add a bit of leeway
> options(future.globals.maxSize = maxSize)
>
> for (chunk.size in c(1L, 2L, 5L, structure(10L, ordering = "random"))) {
+ y <- future_mapply(FUN = FUN, X, future.chunk.size = chunk.size)
+ str(y)
+ stopifnot(all(unlist(y) == maxSize))
+ cat(sprintf("maxSize = %g bytes\nfuture.globals.maxSize = %g bytes\n",
+ maxSize, getOption("future.globals.maxSize")))
+ stopifnot(getOption("future.globals.maxSize") == maxSize)
+ }
num [1:10] 4727 4727 4727 4727 4727 ...
maxSize = 4726.8 bytes
future.globals.maxSize = 4726.8 bytes
num [1:10] 4727 4727 4727 4727 4727 ...
maxSize = 4726.8 bytes
future.globals.maxSize = 4726.8 bytes
num [1:10] 4727 4727 4727 4727 4727 ...
maxSize = 4726.8 bytes
future.globals.maxSize = 4726.8 bytes
Error in getGlobalsAndPackages(expr, envir = envir, tweak = tweakExpression, :
The total size of the 5 globals exported for future expression (‘{; ...future.globals.maxSize.org <- getOption("future.globals.maxSize"); if (!identical(...future.globals.maxSize.org, ...future.globals.maxSize)) {; oopts <- options(future.globals.maxSize = ...future.globals.maxSize); on.exit(options(oopts), add = TRUE); }; ...; do.call(mapply, args = args); }; }’) is 5.33 KiB.. This exceeds the maximum allowed size of 4.62 KiB (option 'future.globals.maxSize'). The three largest globals are ‘...future.elements_ii’ (4.18 KiB of class ‘list’), ‘...future.FUN’ (1.11 KiB of class ‘function’) and ‘...future.globals.maxSize’ (36 bytes of class ‘numeric’)
Calls: future_mapply -> future_xapply -> future -> getGlobalsAndPackages
Execution halted
Reproduce example
We are using this script to run the test suite. If you can run it on i386 you should be able to reproduce the issue.
Expected behavior
I could imagine the following options
- Tests passing on all architectures including 32bit
- Excluding specific tests for 32bit architectures that are known to fail
- Declare future.apply to work on 64bit only
I could adapt the Debian package to these options.
Session information
> sessionInfo()
R version 4.3.2 (2023-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux trixie/sid
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.25.so; LAPACK version 3.11.0
locale:
[1] LC_CTYPE=de_DE.UTF-8 LC_NUMERIC=C
[3] LC_TIME=de_DE.UTF-8 LC_COLLATE=de_DE.UTF-8
[5] LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=de_DE.UTF-8
[7] LC_PAPER=de_DE.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
time zone: Europe/Berlin
tzcode source: system (glibc)
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.3.2
Do you have any idea how to fix this problem? I might exclude this test for 32bit architectures, but I would like to hear your opinion about this.
Hi, thanks for reaching, and sorry for radio silence. I've been and am swamped right now, but this is on my radar and to-do list.
Thanks for putting it on the todo list for next release. Do you have some release schedule in mind? I'm just worried since lots of CRAN packages are affected and these were just removed from Debian testing distribution due to that issue.
I pulled out an old Raspberry Pi to see if I could reproduce this, but I couldn't. So, without a way to reproduce this, I'll have to patch it "in the dark". The https://ci.debian.net/data/autopkgtest/testing/i386/r/r-cran-future.apply/42795227/log.gz are now also gone, so I cannot see if this was the only error, or just the last one.
I've added a bit more leeway to the test that depends on randomness, cf. 9769e46.
Will you be able to check the version in the develop branch to see if this was sufficient?
Here is a fresh test log from Debian, but not with your recent changes https://ci.debian.net/packages/r/r-cran-future.apply/unstable/i386/44417361/
Thanks. From that log file, it looks like that was the only test that failed;
$ wget https://ci.debian.net/data/autopkgtest/unstable/i386/r/r-cran-future.apply/44417361/log.gz
$ zcat log.gz | grep -B 10 -E "FAIL"
198s future.globals.maxSize = 4726.8 bytes
199s num [1:10] 4727 4727 4727 4727 4727 ...
199s maxSize = 4726.8 bytes
199s future.globals.maxSize = 4726.8 bytes
199s Error in getGlobalsAndPackages(expr, envir = envir, tweak = tweakExpression, :
199s The total size of the 5 globals exported for future expression (‘{; ...future.globals.maxSize.org <- getOption("future.globals.maxSize"); if (!identical(...future.globals.maxSize.org, ...future.globals.maxSize)) {; oopts <- options(future.globals.maxSize = ...future.globals.maxSize); on.exit(options(oopts), add = TRUE); }; ...; do.call(mapply, args = args); }; }’) is 5.33 KiB.. This exceeds the maximum allowed size of 4.62 KiB (option 'future.globals.maxSize'). The three largest globals are ‘...future.elements_ii’ (4.18 KiB of class ‘list’), ‘...future.FUN’ (1.11 KiB of class ‘function’) and ‘...future.globals.maxSize’ (36 bytes of class ‘numeric’)
199s Calls: future_mapply -> future_xapply -> future -> getGlobalsAndPackages
199s Execution halted
199s autopkgtest [22:23:49]: test run-unit-test: -----------------------]
199s autopkgtest [22:23:49]: test run-unit-test: - - - - - - - - - - results - - - - - - - - - -
199s run-unit-test FAIL non-zero exit status 1
--
285s Type 'q()' to quit R.
285s
285s > library('future.apply')
285s Loading required package: future
285s >
285s >
286s autopkgtest [22:25:16]: test pkg-r-autopkgtest: -----------------------]
286s autopkgtest [22:25:16]: test pkg-r-autopkgtest: - - - - - - - - - - results - - - - - - - - - -
286s pkg-r-autopkgtest PASS
286s autopkgtest [22:25:16]: @@@@@@@@@@@@@@@@@@@@ summary
286s run-unit-test FAIL non-zero exit status 1I've launched the revdep checks (151 packages) ...
Reproduce example
We are using this script to run the test suite. If you can run it on i386 you should be able to reproduce the issue.
Ah, this not using the R CMD check framework. Note that the run-unit-test script;
#!/bin/sh -e
pkgname=future.apply
debname=r-cran-future.apply
if [ "$AUTOPKGTEST_TMP" = "" ] ; then
AUTOPKGTEST_TMP=`mktemp -d /tmp/${debname}-test.XXXXXX`
trap "rm -rf $AUTOPKGTEST_TMP" 0 INT QUIT ABRT PIPE TERM
fi
cd $AUTOPKGTEST_TMP
cp -a /usr/share/doc/$debname/tests/* $AUTOPKGTEST_TMP
gunzip -r *
for testfile in *.R; do
echo "BEGIN TEST $testfile"
LC_ALL=C.UTF-8 R --no-save < $testfile
doneis not necessarily running the tests the same way as R CMD check. I think it can at best be considered an approximation. I think it also ignores any test files using lower-case filename extensions (*.r). It probably also misses things such as R_DEFAULT_PACKAGES=NULL that R CMD check uses internally. So, that approach will find false positives (like this one here).
I could trigger the error on Raspberry Pi (32-bit) using LC_ALL=C.UTF-8 R --no-save < future_mapply,globals.R.
... but, the way Debian tests packages made me explore more of how R CMD check works, and it turns out I can reproduce the problem on Raspberry Pi (32-bit) using R CMD check without --as-cran (see below), which means R itself is sensitive to differences between R CMD check and R CMD check --as-cran.
R CMD check future.apply_1.11.1-9001.tar.gz
* using log directory ‘/home/pi/repositories/future.apply.Rcheck’
* using R version 4.2.2 Patched (2022-11-10 r83330)
* using platform: arm-unknown-linux-gnueabihf (32-bit)
* using session charset: UTF-8
* checking for file ‘future.apply/DESCRIPTION’ ... OK
* this is package ‘future.apply’ version ‘1.11.1-9001’
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘future.apply’ can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
Running ‘fold.R’
Running ‘future_apply.R’
Running ‘future_by.R’
Running ‘future_eapply.R’
Running ‘future_lapply,RNG.R’
Running ‘future_lapply,globals.R’
Running ‘future_lapply.R’
Running ‘future_mapply,globals.R’
ERROR
Running the tests in ‘tests/future_mapply,globals.R’ failed.
Last 13 lines of output:
+ stopifnot(getOption("future.globals.maxSize") == maxSize)
+ }
num [1:10] 4872 4872 4872 4872 4872 ...
maxSize = 4872 bytes
future.globals.maxSize = 4872 bytes
num [1:10] 4872 4872 4872 4872 4872 ...
maxSize = 4872 bytes
future.globals.maxSize = 4872 bytes
num [1:10] 4872 4872 4872 4872 4872 ...
maxSize = 4872 bytes
future.globals.maxSize = 4872 bytes
Error in getGlobalsAndPackages(expr, envir = envir, tweak = tweakExpression, :
The total size of the 5 globals exported for future expression ('{; ...future.globals.maxSize.org <- getOption("future.globals.maxSize"); if (!identical(...future.globals.maxSize.org, ...future.globals.maxSize)) {; oopts <- options(future.globals.maxSize = ...future.globals.maxSize); on.exit(options(oopts), add = TRUE); }; ...; do.call(mapply, args = args); }; }') is 5.41 KiB.. This exceeds the maximum allowed size of 4.76 KiB (option 'future.globals.maxSize'). The three largest globals are '...future.elements_ii' (4.22 KiB of class 'list'), '...future.FUN' (1.16 KiB of class 'function') and '...future.globals.maxSize' (40 bytes of class 'numeric')
Calls: future_mapply -> future_xapply -> future -> getGlobalsAndPackages
Execution halted
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking running R code from vignettes ...
‘future.apply-1-overview.md.rsp’... OK
NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... WARNING
LaTeX errors when creating PDF version.
This typically indicates Rd problems.
* checking PDF version of manual without index ... ERROR
Re-running with no redirection of stdout/stderr.
Hmm ... looks like a package
Converting parsed Rd's to LaTeX
Creating pdf output from LaTeX ...
Warning in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet, :
texi2dvi script/program not available, using emulation
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet, :
pdflatex is not available
Warning in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet, :
texi2dvi script/program not available, using emulation
Error in texi2dvi(file = file, pdf = TRUE, clean = clean, quiet = quiet, :
pdflatex is not available
Error in running tools::texi2pdf()
You may want to clean up by 'rm -Rf /tmp/RtmpclSt5h/Rd2pdf1e42246564dd'
* DONE
Status: 2 ERRORs, 1 WARNINGActually, scratch what I said in the last comment about R CMD check and R CMD check --as-cran. I can reproduce the error with both. I'm not sure what I've been smoking - maybe I shouldn't try to multitask three different things at the same time.
I've submitted future.apply 1.11.2 to CRAN.
Just to make sure I understand your point about our run-unit-test script correctly: Could you suggest a patch how you would change this script to fit your expectation?
Just to make sure I understand your point about our
run-unit-testscript correctly: Could you suggest a patch how you would change this script to fit your expectation?
I think the only patch that would be 100% true to R CMD check is to run R CMD check.
future.apply 1.11.2 fixing this is now on CRAN