Connection limit error while running OUTRIDER

[pkothiyal]

Hi, I was trying to follow the vignette but ran into an error due to limit on number of connections. I wanted to see if you have any suggestions on how to work around the issue. Thanks.

> ctsFile <- system.file('extdata', 'KremerNBaderSmall.tsv', package='OUTRIDER')
> ctsTable <- read.table(ctsFile, check.names=FALSE)
> ods <- OutriderDataSet(countData=ctsTable)
> ods <- filterExpression(ods, onlyZeros=TRUE, filterGenes=TRUE)
216 genes are filtered out due to zero counts. This is 21.6% of the genes.
> ods <- OUTRIDER(ods)
Fri May 25 13:24:36 2018: SizeFactor estimation ...
Fri May 25 13:24:36 2018: Running auto correct ...
[1] "Initial PCA loss: 4.62018005328588"
[1] "Time elapsed: 7.89964747428894"
[1] "nb-PCA loss: 4.43675707619974"
Fri May 25 13:24:44 2018: Fitting the data ...
Error in .local(x, ...) : 
  cannot create 783 workers; 125 connections available in this session

> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: SUSE Linux Enterprise Server 11 SP4

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] OUTRIDER_0.99.7            GenomicFeatures_1.28.3     AnnotationDbi_1.38.2      
 [4] BiocParallel_1.10.1        RcppArmadillo_0.6.700.6.0  githubinstall_0.2.2       
 [7] devtools_1.13.5            HTSFilter_1.16.0           BiocInstaller_1.26.1      
[10] mixOmics_6.3.1             lattice_0.20-35            MASS_7.3-47               
[13] RColorBrewer_1.1-2         pcaMethods_1.68.0          cowplot_0.8.0             
[16] ggpmisc_0.2.16             ggsignif_0.4.0             ggpubr_0.1.5              
[19] magrittr_1.5               ggplot2_2.2.1              bindrcpp_0.2              
[22] DESeq2_1.16.1              SummarizedExperiment_1.6.3 DelayedArray_0.2.7        
[25] matrixStats_0.52.2         Biobase_2.36.2             GenomicRanges_1.28.3      
[28] GenomeInfoDb_1.12.2        IRanges_2.10.2             S4Vectors_0.14.3          
[31] BiocGenerics_0.22.1        plyr_1.8.4                 edgeR_3.18.1              
[34] limma_3.32.10              data.table_1.11.2         

[c-mertes]

Thanks for pointing this out. I was able to reproduce the error. I think this relates to the issue of having a maximum number of connections one can have open in R (in addition to the three always reserved) is 125.

A quick workaround is to register your own correct BiocParallel Backend.

register(MulticoreParam(10))

Registering more then 125 workers would result in the same error as described above.

For reference HenrikBengtsson/Wishlist-for-R#28
Bioconductor/BiocParallel#55

I also wrote an issue for BiocParallel to create a correct default BiocParallel backend.
Bioconductor/BiocParallel#74

[c-mertes]

@pkothiyal I think this issue can be closed. Martin Morgen implemented a working default for servers with more than 125 cores in the new BiocParallel version 1.15.4 and also if you do not want to upgrade the package you can register your own MulticoreParam as follows:

register(MulticoreParam(10))

[pkothiyal]

Thank you. I will try it out with your suggestion.