MPLCONFIGDIR expected in $HOME outside what singularity allows
Closed this issue · 2 comments
smoe commented
Description of the bug
My home directory is not where the data is located and under -profile singularity the matplotlib config directory is then not available.
I then prefixed the invocation of nextflow with
MPLCONFIGDIR=$(pwd)
but that did not change the error.
Command used and terminal output
$ MPLCONFIGDIR=$(pwd) nextflow run nf-core/scdownstream -profile singularity --input samplesheet.csv --outdir results_20240701 -r dev
-[nf-core/scdownstream] Pipeline completed with errors-
Pulling Singularity image https://depot.galaxyproject.org/singularity/multiqc:1.22.2--pyhdfd78af_0 [cache /data/sm718/singularityCacheDir/depot.galaxyproject.org-singularity-multiqc-1.22.2--pyhdfd7
8af_0.img]
ERROR ~ Error executing process > 'NFCORE_SCDOWNSTREAM:SCDOWNSTREAM:PREPROCESS:ADATA_UNIFY (JM_24h)'
Caused by:
Process `NFCORE_SCDOWNSTREAM:SCDOWNSTREAM:PREPROCESS:ADATA_UNIFY (JM_24h)` terminated with an error exit status (1)
Command executed [/home/sm718/.nextflow/assets/nf-core/scdownstream/./workflows/../subworkflows/local/../../modules/local/adata/unify/templates/unify.py]:
#!/usr/bin/env python3
import scanpy as sc
import scipy
from scipy.sparse import csr_matrix
import platform
....
Command error:
Matplotlib created a temporary cache directory at /tmp/matplotlib-p_bjbrvd because the default path (/home/me/.config/matplotlib) is not a writable directory; it is highly recommended to set t
he MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Traceback (most recent call last):
File ".command.sh", line 3, in <module>
import scanpy as sc
File "/opt/conda/lib/python3.12/site-packages/scanpy/__init__.py", line 45, in <module>
from . import datasets, experimental, external, get, logging, metrics, queries
File "/opt/conda/lib/python3.12/site-packages/scanpy/experimental/__init__.py", line 3, in <module>
from . import pp
File "/opt/conda/lib/python3.12/site-packages/scanpy/experimental/pp/__init__.py", line 3, in <module>
from scanpy.experimental.pp._highly_variable_genes import highly_variable_genes
File "/opt/conda/lib/python3.12/site-packages/scanpy/experimental/pp/_highly_variable_genes.py", line 26, in <module>
from scanpy.preprocessing._distributed import materialize_as_ndarray
File "/opt/conda/lib/python3.12/site-packages/scanpy/preprocessing/__init__.py", line 5, in <module>
from ._deprecated.highly_variable_genes import filter_genes_dispersion
File "/opt/conda/lib/python3.12/site-packages/scanpy/preprocessing/_deprecated/highly_variable_genes.py", line 13, in <module>
from .._utils import _get_mean_var
File "/opt/conda/lib/python3.12/site-packages/scanpy/preprocessing/_utils.py", line 78, in <module>
@numba.njit(cache=True)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/site-packages/numba/core/decorators.py", line 229, in wrapper
disp.enable_caching()
File "/opt/conda/lib/python3.12/site-packages/numba/core/dispatcher.py", line 856, in enable_caching
self._cache = FunctionCache(self.py_func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/site-packages/numba/core/caching.py", line 601, in __init__
self._impl = self._impl_class(py_func)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.12/site-packages/numba/core/caching.py", line 337, in __init__
raise RuntimeError("cannot cache function %r: no locator available "
RuntimeError: cannot cache function 'sparse_mean_var_minor_axis': no locator available for file '/opt/conda/lib/python3.12/site-packages/scanpy/preprocessing/_utils.py'
### Relevant files
_No response_
### System information
N E X T F L O W
version 24.04.2 build 5914
created 29-05-2024 06:19 UTC (08:19 CEST)
cite doi:10.1038/nbt.3820
http://nextflow.io
Version of nf-core/scdownstream is current /dev
CentOS Linux
slurm
nictru commented
Could you please try the following:
- Create a file
nextflow.config
in your launch directory - Add the following line:
singularity.runOptions = '--no-mount tmp --writable-tmpfs'
- Try again
If this works, I can add it to the pipeline configuration
smoe commented
Yes, this works! Thank you tons!