Error executing process > 'parse_kit_metadata (1)' with singularity
Closed this issue · 5 comments
Operating System
Other Linux (please specify below)
Other Linux
18.04.4
Workflow Version
v1.1.0
Workflow Execution
Command line
EPI2ME Version
No response
CLI command run
~/nextflow-23.12.0-edge-all run epi2me-labs/wf-single-cell \
--fastq fastq/ \
--kit_name multiome \
--kit_version v1 \
--expected_cells 5000 \
--ref_genome_dir /home/ubuntu/cellranger/GRCh38-2020-A/ \
--sample $SAMPLE \
--max_threads 20 \
-c openstack.cfg \
-profile singularity \
-resume
Workflow Execution - CLI Execution Profile
singularity
What happened?
I'm trying to use the singularity container on the off chance that somehow fixes #81 (which I don't expect it to, but figured I should give it a chance). Instead I explode very early on in the kit metadata parsing.
I encounter the same problem when trying to do the demo:
~/nextflow-23.12.0-edge-all run epi2me-labs/wf-single-cell \
--fastq wf-single-cell-demo/chr17.fq.gz \
--kit_name 3prime \
--kit_version v3 \
--expected_cells 100 \
--ref_genome_dir wf-single-cell-demo/ \
--plot_umaps \
-profile singularity
I'm on Singularity 3.6.3 in case that matters.
Relevant log output
[14:41:15 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotlib-r1ta80n4 because the default path (/home/ubuntu/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Fontconfig error: No writable cache directories
[14:41:16 - matplotlib.font_manager] generated new fontManager
/home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
@numba.jit()
/home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
@numba.jit()
/home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
@numba.jit()
Traceback (most recent call last):
File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in <module>
cli()
File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 61, in cli
f'{_package_name}.{comp}' for comp in get_components()]
File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 26, in get_components
mod = importlib.import_module(f"{_package_name}.{name}")
File "/home/epi2melabs/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/umap_reduce.py", line 7, in <module>
import umap
File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/__init__.py", line 2, in <module>
from .umap_ import UMAP
File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py", line 41, in <module>
from umap.layouts import (
File "/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py", line 40, in <module>
def rdist(x, y):
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/decorators.py", line 234, in wrapper
disp.enable_caching()
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/dispatcher.py", line 863, in enable_caching
self._cache = FunctionCache(self.py_func)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 601, in __init__
self._impl = self._impl_class(py_func)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/numba/core/caching.py", line 337, in __init__
raise RuntimeError("cannot cache function %r: no locator available "
RuntimeError: cannot cache function 'rdist': no locator available for file '/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py'
Application activity log entry
No response
Were you able to successfully run the latest version of the workflow with the demo data?
no
Other demo data information
No response
For whatever reason I'm not getting the error while quickly checking the demo data on a cluster head node with nextflow-23.04.1-all
and singularity 3.11.5. Think it's also a Bionic Ubuntu so it's not that. Interesting.
Hi @ktpolanski,
I would stay clear of the edge releases of Nextflow in the first instance, and stick with one of the longer term support versions. I don't know that I can comment on that error internal to the umap library.
One thing to note however is that the README is misleading. The correct command to use with the demo data is:
nextflow run epi2me-labs/wf-single-cell \
--expected_cells 100 \
--fastq 'wf-single-cell-demo/chr17.fq.gz' \
--kit_name '3prime' \
--kit_version 'v3' \
--ref_genome_dir 'wf-single-cell-demo' \
--umap_plot_genes 'wf-single-cell-demo/umap_plot_genes.csv'
I apologise the README is incorrect, updating that is currently performed manually. The text if you run
nextflow run epi2me-labs/wf-single-cell -r v1.1.0 --help
is however maintained automatically and in line with our internal automated tests which ensure the demo does successfully complete.
Hi @cjw85,
Running the command you posted, I get the same error with nextflow version 23.10.1 and wf-single-cell v1.1.0 with -profile singularity.
RuntimeError: cannot cache function 'rdist': no locator available for file '/home/epi2melabs/conda/lib/python3.8/site-packages/umap/layouts.py'
Any thoughts?
Best,
Barbara
We've seen this error only whilst using singularity. A solution is to provide a configuration file to Nextflow with the following:
profiles {
singularity {
singularity {
enabled = true
autoMounts = true
runOptions = '--writable-tmpfs'
}
}
}
env {
NUMBA_CACHE_DIR='/path/to/use/for/cache'
}
Not the runOptions declaration here which is not in the workflows included config. You will need to provide a path in the env
section.