ValueError: cannot infer dimensions from zero sized index arrays
OneHitKO opened this issue · 5 comments
OneHitKO commented
Hello, thanks for developing a really nice tool!
I've ran cellbender remove-background with no issues on multiome data while keeping both Gene Expression and Peaks features. However, I'm currently rerunning it on the same dataset with --exclude-feature-types Peaks.
When excluding peaks, I receive the following error after cellbender creates the posterior.h5 file:
cellbender:remove-background: Computing target noise counts per gene for MCKP estimator
cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01
cellbender:remove-background: Computing denoised counts using mckp estimator
cellbender:remove-background: Working on chunk (1/35)
cellbender:remove-background: [1.94 mins per chunk]
cellbender:remove-background: Working on chunk (2/35)
cellbender:remove-background: Working on chunk (3/35)
cellbender:remove-background: Working on chunk (4/35)
cellbender:remove-background: Working on chunk (5/35)
cellbender:remove-background: Working on chunk (6/35)
cellbender:remove-background: Working on chunk (7/35)
cellbender:remove-background: Working on chunk (8/35)
cellbender:remove-background: Working on chunk (9/35)
Traceback (most recent call last):
File "/g/saka/Kristy/software/envs/cellbender/bin/cellbender", line 8, in <module>
sys.exit(main())
File "/home/kou/CellBender/cellbender/base_cli.py", line 118, in main
cli_dict[args.tool].run(args)
File "/home/kou/CellBender/cellbender/remove_background/cli.py", line 185, in run
return main(args)
File "/home/kou/CellBender/cellbender/remove_background/cli.py", line 219, in main
posterior = run_remove_background(args)
File "/home/kou/CellBender/cellbender/remove_background/run.py", line 152, in run_remove_background
file_name=file_name,
File "/home/kou/CellBender/cellbender/remove_background/run.py", line 300, in compute_output_denoised_counts_reports_metrics
use_multiple_processes=args.use_multiprocessing_estimation,
File "/home/kou/CellBender/cellbender/remove_background/posterior.py", line 291, in compute_denoised_counts
**kwargs,
File "/home/kou/CellBender/cellbender/remove_background/estimation.py", line 523, in estimate_noise
noise_log_prob_coo=_subset_coo(noise_log_prob_coo, logic),
File "/home/kou/CellBender/cellbender/remove_background/estimation.py", line 873, in _subset_coo
return sp.coo_matrix((coo.data[logic], (coo.row[logic], coo.col[logic])))
File "/g/saka/Kristy/software/envs/cellbender/lib/python3.7/site-packages/scipy/sparse/coo.py", line 147, in __init__
raise ValueError('cannot infer dimensions from zero '
ValueError: cannot infer dimensions from zero sized index arrays
Here is the exact command:
cellbender remove-background --input cellranger/LN0025/outs/raw_feature_bc_matrix.h5 --output cellbender/LN0025/out_gex.h5 --expected-cells 10000 --exclude-feature-types Peaks --cuda
Any feedback how I can overcome this error, which was not generated before? I'm using CellBender 0.3.1 (Workflow hash e9eeea4787)
Below is the UMI counts knee plot in case it is helpful. Thanks very much for the help!
Wang-Yongqi commented
Hi I've run into the same problem with two different datasets, the problem seems to be also with --exclude-feature-types Peaks.
It always reports error after the 9th chunk here.
Wang-Yongqi commented
Also another line of clue, this issue seems to be Win specific. I successfully ran through on a Mac machine with exactly the same code. Hope it helps.
OneHitKO commented
Interesting, my code fails in Linux, but I can try on MacOS. Thanks!
Baboon61 commented
cellbender 0.3.1 was failing for me as well at the same chunk on Linux with --exclude-feature-types Peaks flag. Downgrading to cellbender 0.3.0 did not raise the issue.
OneHitKO commented
Hello! I recently realized that cellbender 0.3.2 was available. I reran the my code with --exclude-feature-types Peaks but with the same error as initially reported:
cellbender:remove-background: Computing denoised counts using mckp estimator
cellbender:remove-background: Working on chunk (1/29)
cellbender:remove-background: [0.60 mins per chunk]
cellbender:remove-background: Working on chunk (2/29)
cellbender:remove-background: Working on chunk (3/29)
cellbender:remove-background: Working on chunk (4/29)
cellbender:remove-background: Working on chunk (5/29)
cellbender:remove-background: Working on chunk (6/29)
cellbender:remove-background: Working on chunk (7/29)
cellbender:remove-background: Working on chunk (8/29)
cellbender:remove-background: Working on chunk (9/29)
Traceback (most recent call last):
File "/g/saka/Kristy/software/envs/cellbender_0.3.2/bin/cellbender", line 8, in <module>
sys.exit(main())
File "/g/saka/Kristy/software/CellBender/cellbender/base_cli.py", line 118, in main
cli_dict[args.tool].run(args)
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/cli.py", line 193, in run
return main(args)
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/cli.py", line 227, in main
posterior = run_remove_background(args)
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/run.py", line 152, in run_remove_background
file_name=file_name,
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/run.py", line 300, in compute_output_denoised_counts_reports_metrics
use_multiple_processes=args.use_multiprocessing_estimation,
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/posterior.py", line 293, in compute_denoised_counts
**kwargs,
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/estimation.py", line 527, in estimate_noise
noise_log_prob_coo=_subset_coo(noise_log_prob_coo, logic),
File "/g/saka/Kristy/software/CellBender/cellbender/remove_background/estimation.py", line 877, in _subset_coo
return sp.coo_matrix((coo.data[logic], (coo.row[logic], coo.col[logic])))
File "/g/saka/Kristy/software/envs/cellbender_0.3.2/lib/python3.7/site-packages/scipy/sparse/coo.py", line 147, in __init__
raise ValueError('cannot infer dimensions from zero '
ValueError: cannot infer dimensions from zero sized index arrays
It would be great if this error was resolved, as cellbender 0.3.0 gave me negative count values in the filtered matrix for certain samples. Thanks for your help!
Environment details:
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
anndata 0.8.0 pypi_0 pypi
anyio 3.7.1 pypi_0 pypi
argon2-cffi 23.1.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
attrs 24.2.0 pypi_0 pypi
backcall 0.2.0 pypi_0 pypi
beautifulsoup4 4.12.3 pypi_0 pypi
bleach 6.0.0 pypi_0 pypi
ca-certificates 2024.7.4 hbcca054_0 conda-forge
cellbender 0.3.2 pypi_0 pypi
cffi 1.15.1 pypi_0 pypi
click 8.1.7 pypi_0 pypi
comm 0.1.4 pypi_0 pypi
cycler 0.11.0 pypi_0 pypi
debugpy 1.7.0 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
entrypoints 0.4 pypi_0 pypi
exceptiongroup 1.2.2 pypi_0 pypi
fastjsonschema 2.20.0 pypi_0 pypi
fonttools 4.38.0 pypi_0 pypi
h5py 3.8.0 pypi_0 pypi
idna 3.7 pypi_0 pypi
importlib-metadata 6.7.0 pypi_0 pypi
importlib-resources 5.12.0 pypi_0 pypi
ipykernel 6.16.2 pypi_0 pypi
ipython 7.34.0 pypi_0 pypi
ipython-genutils 0.2.0 pypi_0 pypi
ipywidgets 8.1.3 pypi_0 pypi
jedi 0.19.1 pypi_0 pypi
jinja2 3.1.4 pypi_0 pypi
jsonschema 4.17.3 pypi_0 pypi
jupyter 1.0.0 pypi_0 pypi
jupyter-client 7.4.9 pypi_0 pypi
jupyter-console 6.6.3 pypi_0 pypi
jupyter-contrib-core 0.4.2 pypi_0 pypi
jupyter-contrib-nbextensions 0.7.0 pypi_0 pypi
jupyter-core 4.12.0 pypi_0 pypi
jupyter-highlight-selected-word 0.2.0 pypi_0 pypi
jupyter-nbextensions-configurator 0.6.4 pypi_0 pypi
jupyter-server 1.24.0 pypi_0 pypi
jupyterlab-pygments 0.2.2 pypi_0 pypi
jupyterlab-widgets 3.0.11 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
ld_impl_linux-64 2.40 hf3520f5_7 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 14.1.0 h77fa898_0 conda-forge
libgomp 14.1.0 h77fa898_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libsqlite 3.46.0 hde9e2c9_0 conda-forge
libstdcxx-ng 14.1.0 hc0a3c3a_0 conda-forge
libzlib 1.3.1 h4ab18f5_1 conda-forge
llvmlite 0.39.1 pypi_0 pypi
loompy 3.0.7 pypi_0 pypi
lxml 5.3.0 pypi_0 pypi
lxml-html-clean 0.2.0 pypi_0 pypi
markupsafe 2.1.5 pypi_0 pypi
matplotlib 3.5.3 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mistune 0.8.4 pypi_0 pypi
natsort 8.4.0 pypi_0 pypi
nbclassic 1.1.0 pypi_0 pypi
nbclient 0.7.4 pypi_0 pypi
nbconvert 6.5.4 pypi_0 pypi
nbformat 5.8.0 pypi_0 pypi
ncurses 6.5 h59595ed_0 conda-forge
nest-asyncio 1.6.0 pypi_0 pypi
notebook 6.5.7 pypi_0 pypi
notebook-shim 0.2.4 pypi_0 pypi
numba 0.56.4 pypi_0 pypi
numexpr 2.8.6 pypi_0 pypi
numpy 1.21.6 pypi_0 pypi
numpy-groupies 0.9.22 pypi_0 pypi
nvidia-cublas-cu11 11.10.3.66 pypi_0 pypi
nvidia-cuda-nvrtc-cu11 11.7.99 pypi_0 pypi
nvidia-cuda-runtime-cu11 11.7.99 pypi_0 pypi
nvidia-cudnn-cu11 8.5.0.96 pypi_0 pypi
openssl 3.3.1 h4bc722e_2 conda-forge
opt-einsum 3.3.0 pypi_0 pypi
packaging 24.0 pypi_0 pypi
pandas 1.3.5 pypi_0 pypi
pandocfilters 1.5.1 pypi_0 pypi
parso 0.8.4 pypi_0 pypi
pexpect 4.9.0 pypi_0 pypi
pickleshare 0.7.5 pypi_0 pypi
pillow 9.5.0 pypi_0 pypi
pip 24.0 pyhd8ed1ab_0 conda-forge
pkgutil-resolve-name 1.3.10 pypi_0 pypi
prometheus-client 0.17.1 pypi_0 pypi
prompt-toolkit 3.0.47 pypi_0 pypi
psutil 6.0.0 pypi_0 pypi
ptyprocess 0.7.0 pypi_0 pypi
pycparser 2.21 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pyparsing 3.1.2 pypi_0 pypi
pyro-api 0.1.2 pypi_0 pypi
pyro-ppl 1.8.6 pypi_0 pypi
pyrsistent 0.19.3 pypi_0 pypi
python 3.7.12 hf930737_100_cpython conda-forge
python-dateutil 2.9.0.post0 pypi_0 pypi
pytz 2024.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
pyzmq 26.1.0 pypi_0 pypi
qtconsole 5.4.4 pypi_0 pypi
qtpy 2.4.1 pypi_0 pypi
readline 8.2 h8228510_1 conda-forge
scipy 1.7.3 pypi_0 pypi
send2trash 1.8.3 pypi_0 pypi
setuptools 65.6.3 pyhd8ed1ab_0 conda-forge
six 1.16.0 pypi_0 pypi
sniffio 1.3.1 pypi_0 pypi
soupsieve 2.4.1 pypi_0 pypi
sqlite 3.46.0 h6d4b2fc_0 conda-forge
tables 3.7.0 pypi_0 pypi
terminado 0.17.1 pypi_0 pypi
tinycss2 1.2.1 pypi_0 pypi
tk 8.6.13 noxft_h4845f30_101 conda-forge
torch 1.13.1 pypi_0 pypi
tornado 6.2 pypi_0 pypi
tqdm 4.66.5 pypi_0 pypi
traitlets 5.9.0 pypi_0 pypi
typing-extensions 4.7.1 pypi_0 pypi
wcwidth 0.2.13 pypi_0 pypi
webencodings 0.5.1 pypi_0 pypi
websocket-client 1.6.1 pypi_0 pypi
wheel 0.34.2 py37_0 conda-forge
widgetsnbextension 4.0.11 pypi_0 pypi
xz 5.2.6 h166bdaf_0 conda-forge
zipp 3.15.0 pypi_0 pypi