thierrygosselin/grur

Package dependencies

IdoBar opened this issue · 5 comments

IdoBar commented

Hi Thierry,

As I've mentioned in my other Issue, grur requires quite a lot of dependencies, which in my case raised an maximal number of DLLs reached error when loaded with a few other packages.
It seems that the issue is with the Rdynload.c of the base R code: #define MAX_NUM_DLLS 100.
In R versions >3.4, you can set a different max number of DLLs using and environmental variable R_MAX_NUM_DLLS. (taken from this SO thread)
From the release notes:

The maximum number of DLLs that can be loaded into R e.g. via dyn.load() can now be increased by setting the environment variable R_MAX_NUM_DLLS before starting R.

I'm pretty sure some of the packages are not entirely needed, such as maps, mapdata, and non-standard tidyverse packages, such as glue, tidyselect, purrrlyr, reshape2 (isn't it a part of tidyr now?), plyr (mostly replaced by dplyr), etc.

Thanks, Ido

thierrygosselin commented

Hi Ido,

As stated in here I have tried with fresh Linux and Mac and I can't reproduce your problem. I'll give it a try again, but for now you're the only one reporting this issue. And I had never heard of this before, so it's all new to me. Have you read somewhere that it could be computer/Os specific ?

Inside grur I'm always using :: for package functions.
During grur install, it will need to install all the packages in the Imports field of the DESCRIPTION file as well as packages required by those packages, this is where it hurts.

However it doesn't load these packages dependencies unless needed
Here is the list for grur dependencies:

    dplyr (>= 0.7.0),
    purrr,
    readr,
    tibble,
    tidyr,
    ggplot2,
    vcfR (>= 1.3.0),
    data.table,
    stringi,
    ape,
    parallel,
    amap,
    magrittr,
    stats,
    utils,
    ranger,
    pbmcapply,
    xgboost,
    randomForestSRC,
    pegas,
    radiator,
    fst (>= 0.7.2),
    purrrlyr,
    strataG,
    rmetasim,
    rlang

I don't see packages I could remove.
I've seen packages with a lot more dependencies.
Sadly, tidyverse cannot be used in the DESCRIPTION file during package development.
As of maps, mapdata, glue, tidyselect, reshape2 and plyr they are not used by grur
so I cannot do anything about it, except suggesting authors of packages using those to remove it from their dependencies...

So first step for me is trying to reproduce the issue with DLLs in order to fix it...
I'll reopen the issue if I have a solution.
Thanks!
Thierry

thierrygosselin commented

Did a couple of tests...

With a clean R session, devtools::session_info() gives me 14 packages

After library(adegenet), devtools::session_info() gives me 69 packages

restart R
library(strataG), devtools::session_info() gives me 115 packages

restart R
library(radiator), devtools::session_info() gives me 51 packages

restart R
library(grur), devtools::session_info() gives me 156 packages !!! Wo...

thierrygosselin commented

I can reduce from 156 to 114 if I put strataG in the Suggests field of the DESCRIPTION file.

EricArcher commented
thierrygosselin commented

Recently, I've also put: lightgbm, missRanger, randomForestSRC, ranger, rmetasim, xgboost in the Suggests field to reduce dependencies during installation.

The sad part is that strataG is not installed automatically with grur,
If user try to run grur::simulate_rad without strataG on their system
it stops and ask to install it:

if (!requireNamespace("strataG", quietly = TRUE)) {
    stop("strataG needed for this function to work
         Install with install.packages('strataG')", call. = FALSE)
  }

I'm updating doc to give an example of simulation

@EricArcher and @IdoBar Other suggestions to remove potential DLL problem on linux machines ?