ERROR; return code from pthread_create() is 22 when calling preprocessCore::normalize.quantiles()
lgatto opened this issue · 12 comments
Describe the bug
Running preprocessCore::normalize.quantiles() generates the following error using bioconductor/bioconductor_docker:devel
Error: processing vignette 'v02-QFeaturesPipeline.Rmd' failed with diagnostics:
ERROR; return code from pthread_create() is 22
See here for a full check.
To Reproduce
Steps to reproduce the behavior:
- This run, that uses quantile normalisation produces the error above.
- This run, using a different normalisation (that doesn't use
preprocessCore::normalize.quantiles()) succeeds. The only difference between these two can be found here.
Expected behavior
The expected behaviour would be for the normalize() call to succeed, like in case 1 above. The code in case 2 above works locally.
Additional context
The following post suggest that it might be related to libopenblas.
This is a follow up from a question that I originally posted on slack (that I thought was sorted out).
The following manual installation of preprocessCore fixes the issue
- name: Manually install preprocessCore
run: |
git clone https://github.com/bmbolstad/preprocessCore.git
cd preprocessCore
R CMD INSTALL --configure-args="--disable-threading" .
The following manual installation of
preprocessCorefixes the issue- name: Manually install preprocessCore run: | git clone https://github.com/bmbolstad/preprocessCore.git cd preprocessCore R CMD INSTALL --configure-args="--disable-threading" .
Hello, how do I manually install it on the server? Because I do not know much about the operation of the server, I hope to get a reply. Thank you very much!
The following manual installation of
preprocessCorefixes the issue- name: Manually install preprocessCore run: | git clone https://github.com/bmbolstad/preprocessCore.git cd preprocessCore R CMD INSTALL --configure-args="--disable-threading" .
I've tried installing the package by cloning this git or with BiocManager, but neither was successful. Are there any other possible reasons causing this problem?
@lgatto I'm having the same issue - can you print a set of package versions which you use and with which this now works? I'm currently observing the same issue in a package I'm working on.
same issue here, was not resolved by manually installing preprocessCore using either git clone or BiocManager. downgraded openblas per links in above subthreads, and that worked. i'm using conda so this was what worked for me:
conda install -c conda-forge openblas=0.3.3
Hey @annchenknodt, thank you for the report and the posible solution. Could you please provide a reproducible small example of the issue so that I may investigate and if necessary downgrade the pre-existing version of openblas by default if that is the source of the issue?
I'm not sure how to provide a small reproducible examplem as the dataset is massive and this is my very first time working with these tools (or methylation data in general).... I am just following instructions for running analyses as a participating site in a large consortium. Hopefully my session info and relevant lines of code are helpful:
`> sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: x86_64-conda-linux-gnu (64-bit)
Running under: CentOS Stream 8
Matrix products: default
BLAS/LAPACK: /hpc/home/ark19/miniconda3/envs/ENIGMAepi/lib/libopenblasp-r0.3.21.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets
[8] methods base
other attached packages:
[1] preprocessCore_1.60.2
[2] IlluminaHumanMethylationEPICanno.ilm10b4.hg19_0.6.0
[3] corpcor_1.6.10
[4] IlluminaHumanMethylationEPICmanifest_0.3.0
[5] FlowSorted.Blood.450k_1.36.0
[6] limma_3.54.2
[7] RColorBrewer_1.1-3
[8] minfiData_0.44.0
[9] IlluminaHumanMethylation450kanno.ilmn12.hg19_0.6.1
[10] IlluminaHumanMethylation450kmanifest_0.4.0
[11] minfi_1.44.0
[12] bumphunter_1.40.0
[13] locfit_1.5-9.8
[14] iterators_1.0.14
[15] foreach_1.5.2
[16] Biostrings_2.66.0
[17] XVector_0.38.0
[18] SummarizedExperiment_1.28.0
[19] Biobase_2.58.0
[20] MatrixGenerics_1.10.0
[21] matrixStats_1.0.0
[22] GenomicRanges_1.50.2
[23] GenomeInfoDb_1.34.9
[24] IRanges_2.32.0
[25] S4Vectors_0.36.2
[26] BiocGenerics_0.44.0
loaded via a namespace (and not attached):
[1] rjson_0.2.21 ellipsis_0.3.2
[3] siggenes_1.72.0 mclust_6.0.0
[5] base64_2.0.1 bit64_4.0.5
[7] AnnotationDbi_1.60.2 fansi_1.0.3
[9] xml2_1.3.5 codetools_0.2-19
[11] splines_4.2.0 sparseMatrixStats_1.10.0
[13] cachem_1.0.8 scrime_1.3.5
[15] Rsamtools_2.14.0 annotate_1.76.0
[17] dbplyr_2.1.1 png_0.1-8
[19] HDF5Array_1.26.0 BiocManager_1.30.22
[21] readr_2.1.4 compiler_4.2.0
[23] httr_1.4.7 assertthat_0.2.1
[25] Matrix_1.6-1 fastmap_1.1.1
[27] cli_3.6.1 prettyunits_1.1.1
[29] tools_4.2.0 glue_1.6.2
[31] GenomeInfoDbData_1.2.9 dplyr_1.0.9
[33] rappdirs_0.3.3 doRNG_1.8.6
[35] Rcpp_1.0.11 vctrs_0.6.3
[37] rhdf5filters_1.10.1 multtest_2.54.0
[39] nlme_3.1-163 rtracklayer_1.58.0
[41] DelayedMatrixStats_1.20.0 stringr_1.4.0
[43] lifecycle_1.0.3 restfulr_0.0.15
[45] rngtools_1.5.2 XML_3.99-0.9
[47] beanplot_1.3.1 zlibbioc_1.44.0
[49] MASS_7.3-60 hms_1.1.3
[51] rhdf5_2.42.1 GEOquery_2.66.0
[53] yaml_2.3.7 curl_5.0.2
[55] memoise_2.0.1 biomaRt_2.54.1
[57] reshape_0.8.9 stringi_1.7.6
[59] RSQLite_2.3.1 genefilter_1.80.3
[61] BiocIO_1.8.0 GenomicFeatures_1.50.4
[63] filelock_1.0.2 BiocParallel_1.32.6
[65] rlang_1.1.1 pkgconfig_2.0.3
[67] bitops_1.0-7 nor1mix_1.3-0
[69] lattice_0.21-8 purrr_0.3.4
[71] Rhdf5lib_1.20.0 GenomicAlignments_1.34.1
[73] bit_4.0.5 tidyselect_1.1.2
[75] plyr_1.8.7 magrittr_2.0.3
[77] R6_2.5.1 generics_0.1.2
[79] DelayedArray_0.24.0 DBI_1.1.3
[81] pillar_1.7.0 survival_3.5-7
[83] KEGGREST_1.38.0 RCurl_1.98-1.12
[85] tibble_3.1.7 crayon_1.5.1
[87] utf8_1.2.2 BiocFileCache_2.6.1
[89] tzdb_0.3.0 progress_1.2.2
[91] grid_4.2.0 data.table_1.14.8
[93] blob_1.2.4 digest_0.6.33
[95] xtable_1.8-4 tidyr_1.2.0
[97] illuminaio_0.40.0 openssl_2.0.2
[99] askpass_1.1 quadprog_1.5-8
`
`> RGset = read.metharray.exp(base = datadir, targets=pd, verbose=TRUE)
object=preprocessQuantile(RGset, fixOutliers = TRUE, removeBadSamples = TRUE, badSampleCutoff = 10.5, quantileNormalize = TRUE, stratified = TRUE, mergeManifest = FALSE, sex = NULL, verbose = TRUE) #Note this step removes samples with low intensity
[preprocessQuantile] Mapping to genome.
[preprocessQuantile] Fixing outliers.
[preprocessQuantile] Quantile normalizing.
Error in normalize.quantiles(mat[Index2, ]) :
ERROR; return code from pthread_create() is 22
In addition: Warning messages:
1: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
2: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
3: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
4: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
5: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
6: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
7: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
8: useNames = NA is deprecated. Instead, specify either useNames = TRUE or useNames = TRUE.
`
While I could not reproduce your solution as there is no conda by default in the container, I was able to codify the previous solution within the container, and it does seem to fix the issue. I have opened #84 which is pending review from colleagues.
In the meantime, here is a piece of R code that might be able to install a working preprocessCore in existing containers:
BiocManager::install("preprocessCore", configure.args = c(preprocessCore = "--disable-threading"), force= TRUE, update=TRUE, type = "source")
Hope this helps!
Thank you @lgatto for reporting the issue and a solution!
I will close this as it was merged in devel in anticipation for 3.18 release. Please feel free to re-open if it's still an issue in the new container.
As of 3.18 and preprocessCore v 1.64.0, I'm still getting this error when I install from source on Linux, regardless of whether it's quantile normalization or rma.
Hey @rmflight sorry for the late reply, just going through GitHub notifications now. Can you please indicate whether you tried installing from source with the above commands? i.e. BiocManager::install("preprocessCore", configure.args = c(preprocessCore = "--disable-threading"), force= TRUE, update=TRUE, type = "source") ? This was the workaround used for the container. If you're experiencing the issue outside of the container, I would also suggest you open an issue at the package repository (https://github.com/bmbolstad/preprocessCore) as it is likely a package-level rather than container-level issue.
Yes, that does seem to work outside the container as well.