Error in running run_QTL_analysis.py
Closed this issue · 3 comments
Hello,
I am trying to run run_QTL_analysis.py on the data in the "test_data" folder and got some error info below. I am wondering if you could help suggest how to fix the error.
The info:
None
anaconda3/lib/python3.7/site-packages/sklearn/utils/deprecation.py:66: DeprecationWarning: Class Imputer is deprecated; Imputer was deprecated in version 0.20 and will be removed in 0.22. Import impute.SimpleImputer from sklearn instead.
warnings.warn(msg, category=DeprecationWarning)
Running QTL analysis.
Intersecting data.
Dropped: 0 samples because they are not present in the genotype file.
Dropped: 0 samples because they are not present in the phenotype file.
Dropped: 0 samples because they are not present in the kinship file.
Dropped: 0 samples because they are not present in the covariate file.
Number of samples with genotype & phenotype data: 90
Number of features to be tested: 19
Total number of variants to be considered, before variante QC and feature intersection: 3204
For feature: 1/19 (ENSG00000253022): 72 SNPs need to be tested.
Please stand by.
None
anaconda3/lib/python3.7/site-packages/tables/table.py:1001: PerformanceWarning: table /ENSG00000253022 is exceeding the recommended maximum number of columns (512); be ready to see PyTables asking for lots of memory and possibly slow I/O
PerformanceWarning)
Traceback (most recent call last):
File "limix_qtl/Limix_QTL/qtl_output.py", line 90, in add_permutation_results_df
table = self.h5file.get_node('/'+feature_id)
File "anaconda3/lib/python3.7/site-packages/tables/file.py", line 1651, in get_node
node = self._get_node(nodepath)
File "anaconda3/lib/python3.7/site-packages/tables/file.py", line 1599, in _get_node
node = self._node_manager.get_node(nodepath)
File "anaconda3/lib/python3.7/site-packages/tables/file.py", line 437, in get_node
node = self.node_factory(key)
File "anaconda3/lib/python3.7/site-packages/tables/group.py", line 1181, in _g_load_child
node_type = self._g_check_has_child(childname)
File "anaconda3/lib/python3.7/site-packages/tables/group.py", line 398, in _g_check_has_child
% (self._v_pathname, name))
tables.exceptions.NoSuchNodeError: group / does not have a child named /ENSG00000253022
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "limix_qtl/Limix_QTL/run_QTL_analysis.py", line 836, in
randomeff_filename=randeff_file, sample_mapping_filename=samplemap_file, extended_anno_filename=extended_anno_file, regressCovariatesUpfront = regressBefore, debugger= debugger)
File "limix_qtl/Limix_QTL/run_QTL_analysis.py", line 645, in run_QTL_analysis
permutation_writer.add_permutation_results_df(perm_df,feature_id)
File "limix_qtl/Limix_QTL/qtl_output.py", line 96, in add_permutation_results_df
"Permutation analysis results")
File "anaconda3/lib/python3.7/site-packages/tables/file.py", line 1066, in create_table
track_times=track_times)
File "anaconda3/lib/python3.7/site-packages/tables/table.py", line 841, in init
byteorder, _log, track_times)
File "anaconda3/lib/python3.7/site-packages/tables/leaf.py", line 290, in init
super(Leaf, self).init(parentnode, name, _log)
File "anaconda3/lib/python3.7/site-packages/tables/node.py", line 266, in init
self._v_objectid = self._g_create()
File "anaconda3/lib/python3.7/site-packages/tables/table.py", line 1028, in _g_create
self._v_new_title, self.filters.complib or '', obversion)
File "tables/tableextension.pyx", line 211, in tables.tableextension.Table._create_table
tables.exceptions.HDF5ExtError: Problems creating the table
Closing remaining open files:test_data/output/qtl_results_1.h5...donetest_data/output/perm_results_1.h5...done
Thanks a lot!
Jun
Hi Marc,
Sorry for the late reply! Here are my package info:
packages in environment at /software/anaconda3/envs/limix_qtl:
Name Version Build Channel
_anaconda_depends 2020.07 py37_0 anaconda
_libgcc_mutex 0.1 main
alabaster 0.7.12 py37_0
almosthere 1.0.15 h46ee950_0 conda-forge
anaconda custom py37_1
anaconda-client 1.7.2 py37_0
anaconda-project 0.8.4 py_0
argh 0.26.2 py37_0
argon2-cffi 20.1.0 py37h7b6447c_1
asn1crypto 1.4.0 py_0
astroid 2.4.2 py37_0
astropy 4.0.2 py37h7b6447c_0
async_generator 1.10 py37h28b3542_0
atomicwrites 1.4.0 py_0
attrs 20.3.0 pyhd3eb1b0_0
autopep8 1.5.4 py_0
babel 2.8.1 pyhd3eb1b0_0
backcall 0.2.0 py_0
backports 1.0 py_2
backports.shutil_get_terminal_size 1.0.0 py37_2
beautifulsoup4 4.9.3 pyhb0f4dca_0
bgen 3.0.3 h256d785_1 conda-forge
bgen-reader 3.0.7 py37h516909a_0 conda-forge
bitarray 1.6.1 py37h27cfd23_0
bkcharts 0.2 py37_0
blas 1.0 mkl
bleach 3.2.1 py_0
blosc 1.20.1 hd408876_0
bokeh 2.2.3 py37_0
boto 2.49.0 py37_0
bottleneck 1.3.2 py37heb32a55_1
brotlipy 0.7.0 py37h7b6447c_1000
bzip2 1.0.8 h7b6447c_0
ca-certificates 2021.5.30 ha878542_0 conda-forge
cachetools 4.2.1 pyhd8ed1ab_0 conda-forge
cairo 1.14.12 h8948797_3
certifi 2021.5.30 py37h89c1867_0 conda-forge
cffi 1.14.3 py37he30daa8_0
chardet 3.0.4 py37_1003 conda-forge
click 7.1.2 py_0
cloudpickle 1.6.0 py_0
clyent 1.2.2 py37_1
colorama 0.4.4 py_0
contextlib2 0.6.0.post1 py_0
cryptography 3.1.1 py37h1ba5d50_0
curl 7.71.1 hbc83047_1
cycler 0.10.0 py37_0
cython 0.29.21 py37he6710b0_0
cytoolz 0.11.0 py37h7b6447c_0
dask 2.30.0 py_0
dask-core 2.30.0 py_0
dbus 1.13.18 hb2f20db_0
decorator 4.4.2 py_0
defusedxml 0.6.0 py_0
diff-match-patch 20200713 py_0
distributed 2.30.1 py37h06a4308_0
docutils 0.16 py37_1
entrypoints 0.3 py37_0
et_xmlfile 1.0.1 py_1001
expat 2.2.10 he6710b0_2
fastcache 1.1.0 py37h7b6447c_0
filelock 3.0.12 py_0
flake8 3.8.4 py_0
flask 1.1.2 py_0
fontconfig 2.13.0 h9420a91_0
freetype 2.10.4 h5ab3b9f_0
fribidi 1.0.10 h7b6447c_0
fsspec 0.8.3 py_0
future 0.18.2 py37_1
get_terminal_size 1.0.0 haa9412d_0
gevent 20.9.0 py37h7b6447c_0
glib 2.66.1 h92f7085_0
glob2 0.7 py_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py37h10f8cd9_2
graphite2 1.3.14 h23475e2_0
greenlet 0.4.17 py37h7b6447c_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb31296c_0
h5py 2.10.0 py37h7918eee_0
harfbuzz 2.4.0 hca77d97_1
hdf5 1.10.4 hb1b8bf9_0
heapdict 1.0.1 py_0
html5lib 1.1 py_0
icu 58.2 he6710b0_3
idna 2.10 py_0
imageio 2.9.0 py_0
imagesize 1.2.0 py_0
importlib-metadata 2.0.0 py_1
importlib_metadata 2.0.0 1
iniconfig 1.1.1 py_0
intel-openmp 2020.2 254
intervaltree 3.1.0 py_0
ipykernel 5.3.4 py37h5ca1d4c_0
ipython 7.19.0 py37hb070fc8_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.5.1 py_1
isort 5.6.4 py_0
itsdangerous 1.1.0 py37_0
jbig 2.1 hdba287a_0
jdcal 1.4.1 py_0
jedi 0.17.1 py37_0
jeepney 0.5.0 pyhd3eb1b0_0
jinja2 2.11.2 py_0
joblib 0.17.0 py_0
jpeg 9b h024ee3a_2
json5 0.9.5 py_0
jsonschema 3.2.0 py_2
jupyter 1.0.0 py37_7
jupyter_client 6.1.7 py_0
jupyter_console 6.2.0 py_0
jupyter_core 4.6.3 py37_0 conda-forge
jupyterlab 2.2.6 py_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 1.2.0 py_0
keyring 21.4.0 py37_1
kiwisolver 1.3.0 py37h2531618_0
krb5 1.18.2 h173b8e3_0
lazy-object-proxy 1.4.3 py37h7b6447c_0
lcms2 2.11 h396b838_0
ld_impl_linux-64 2.33.1 h53a641e_7
libarchive 3.4.2 h62408e4_0
libcurl 7.71.1 h20c2e04_1
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
liblief 0.10.1 he6710b0_0
libllvm10 10.0.1 hbcb73fb_5
libllvm9 9.0.1 h4a3c616_1 anaconda
libpng 1.6.37 hbc83047_0
libsodium 1.0.18 h7b6447c_0
libspatialindex 1.9.3 he6710b0_0
libssh2 1.9.0 h1ba5d50_1
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
libtool 2.4.6 h7b6447c_1005
libuuid 1.0.3 h1bed415_2
libxcb 1.14 h7b6447c_0
libxml2 2.9.10 hb55368b_3
libxslt 1.1.34 hc22bd24_0
llvmlite 0.34.0 py37h269e1b5_4
locket 0.2.0 py37_1
lxml 4.6.1 py37hefd8a0e_0
lz4-c 1.9.2 heb0550a_3
lzo 2.10 h7b6447c_2
markupsafe 1.1.1 py37h14c3975_1
matplotlib 3.3.2 0
matplotlib-base 3.3.2 py37h817c723_0
mccabe 0.6.1 py37_1
mistune 0.8.4 py37h14c3975_1001
mkl 2020.2 256
mkl-service 2.3.0 py37he904b0f_0
mkl_fft 1.2.0 py37h23d657b_0
mkl_random 1.1.1 py37h0573a6f_0
mock 4.0.2 py_0
more-itertools 8.6.0 pyhd3eb1b0_0
mpc 1.1.0 h10f8cd9_1
mpfr 4.0.2 hb69a4c5_1
mpmath 1.1.0 py37_0
msgpack-python 1.0.0 py37hfd86e86_1
multipledispatch 0.6.0 py37_0
nbclient 0.5.1 py_0
nbconvert 6.0.7 py37_0
nbformat 5.0.8 py_0
ncurses 6.2 he6710b0_1
nest-asyncio 1.4.2 pyhd3eb1b0_0
networkx 2.5 py_0
nltk 3.5 py_0
nose 1.3.7 py37_1004
notebook 6.1.4 py37_0
numba 0.51.2 py37h04863e7_1
numexpr 2.7.1 py37h423224d_0
numpy 1.19.2 py37h54aff64_0
numpy-base 1.19.2 py37hfa32c7d_0
numpydoc 1.1.0 pyhd3eb1b0_1
olefile 0.46 py37_0
openpyxl 3.0.5 py_0
openssl 1.1.1h h516909a_0 conda-forge
packaging 20.4 py_0
pandas 1.1.3 py37he6710b0_0
pandoc 2.11 hb0f4dca_0
pandocfilters 1.4.3 py37h06a4308_1
pango 1.45.3 hd140c19_0
parso 0.7.0 py_0
partd 1.1.0 py_0
patchelf 0.12 he6710b0_0
path 15.0.0 py37_0
path.py 12.5.0 0
pathlib2 2.3.5 py37_1
pathtools 0.1.2 py_1
patsy 0.5.1 py37_0
pcre 8.44 he6710b0_0
pep8 1.7.1 py37_0
pexpect 4.8.0 py37_1
pickleshare 0.7.5 py37_1001
pillow 8.0.1 py37he98fc37_0
pip 20.2.4 py37h06a4308_0
pixman 0.40.0 h7b6447c_0
pkginfo 1.6.1 py37h06a4308_0
pluggy 0.13.1 py37_0
ply 3.11 py37_0
prometheus_client 0.8.0 py_0
prompt-toolkit 3.0.8 py_0
prompt_toolkit 3.0.8 0
psutil 5.7.2 py37h7b6447c_0
ptyprocess 0.6.0 py37_0 conda-forge
py 1.9.0 py_0
py-lief 0.10.1 py37h403a769_0
pycodestyle 2.6.0 py_0
pycosat 0.6.3 py37h7b6447c_0
pycparser 2.20 py_2
pycrypto 2.6.1 py37h7b6447c_10
pycurl 7.43.0.6 py37h1ba5d50_0
pydocstyle 5.1.1 py_0
pyflakes 2.2.0 py_0
pygments 2.7.2 pyhd3eb1b0_0
pylint 2.6.0 py37_0
pyodbc 4.0.30 py37he6710b0_0
pyopenssl 19.1.0 py_1
pyparsing 2.4.7 py_0
pyqt 5.9.2 py37h05f1152_2
pyrsistent 0.17.3 py37h7b6447c_0
pysocks 1.7.1 py37_1
pytables 3.6.1 py37h71ec239_0 anaconda
pytest 6.1.1 py37_0 anaconda
pytest-runner 5.3.0 pyh44b312d_0 conda-forge
python 3.7.9 h7579374_0
python-dateutil 2.8.1 py_0
python-jsonrpc-server 0.4.0 py_0
python-language-server 0.35.1 py_0
python-libarchive-c 2.9 py_0
python_abi 3.7 2_cp37m conda-forge
pytz 2020.1 py_0
pywavelets 1.1.1 py37h7b6447c_2
pyxdg 0.27 pyhd3eb1b0_0
pyyaml 5.3.1 py37h7b6447c_1
pyzmq 19.0.2 py37he6710b0_1
qdarkstyle 2.8.1 py_0
qt 5.9.7 h5867ecd_1
qtawesome 1.0.1 py_0
qtconsole 4.7.7 py_0
qtpy 1.9.0 py_0
readline 8.0 h7b6447c_0
regex 2020.10.15 py37h7b6447c_0
requests 2.24.0 py_0
ripgrep 12.1.1 0
rope 0.18.0 py_0
rtree 0.9.4 py37_1
ruamel_yaml 0.15.87 py37h7b6447c_1
scikit-image 0.17.2 py37hdf5156a_0
scikit-learn 0.23.2 py37h0573a6f_0
scipy 1.5.2 py37h0b6359f_0
seaborn 0.11.0 py_0
secretstorage 3.1.2 py37_1
send2trash 1.5.0 py37_0
setuptools 50.3.1 py37h06a4308_1
simplegeneric 0.8.1 py37_2
singledispatch 3.4.0.3 py_1001
sip 4.19.8 py37hf484d3e_0
six 1.15.0 py37h06a4308_0
snappy 1.1.8 he6710b0_0 anaconda
snowballstemmer 2.0.0 py_0
sortedcollections 1.2.1 py_0
sortedcontainers 2.2.2 py_0
soupsieve 2.0.1 py_0
sphinx 3.2.1 py_0
sphinxcontrib 1.0 py37_1
sphinxcontrib-applehelp 1.0.2 py_0
sphinxcontrib-devhelp 1.0.2 py_0
sphinxcontrib-htmlhelp 1.0.3 py_0
sphinxcontrib-jsmath 1.0.1 py_0
sphinxcontrib-qthelp 1.0.3 py_0
sphinxcontrib-serializinghtml 1.1.4 py_0
sphinxcontrib-websupport 1.2.4 py_0
spyder 4.1.5 py37_0
spyder-kernels 1.9.4 py37_0
sqlalchemy 1.3.20 py37h27cfd23_0
sqlite 3.33.0 h62c20be_0
statsmodels 0.12.0 py37h7b6447c_0
sympy 1.6.2 py37h06a4308_1
tbb 2020.3 hfd86e86_0 anaconda
tblib 1.7.0 py_0
terminado 0.9.1 py37_0
testpath 0.4.4 py_0
texttable 1.6.3 pyh9f0ad1d_0 conda-forge
threadpoolctl 2.1.0 pyh5ca1d4c_0
tifffile 2020.10.1 py37hdd07704_2
tk 8.6.10 hbc83047_0
toml 0.10.1 py_0
toolz 0.11.1 py_0
tornado 6.0.4 py37h7b6447c_1
tqdm 4.50.2 py_0
traitlets 5.0.5 py_0
typed-ast 1.4.1 py37h7b6447c_0
typing_extensions 3.7.4.3 py_0
ujson 4.0.1 py37he6710b0_0
unicodecsv 0.14.1 py37_0
unixodbc 2.3.9 h7b6447c_0
urllib3 1.25.11 py_0
watchdog 0.10.3 py37_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py37_1
werkzeug 1.0.1 py_0
wheel 0.35.1 py_0
widgetsnbextension 3.5.1 py37_0 conda-forge
wrapt 1.11.2 py37h7b6447c_0
wurlitzer 2.0.1 py37_0
xarray 0.17.0 pyhd8ed1ab_0 conda-forge
xlrd 1.2.0 py37_0
xlsxwriter 1.3.7 py_0
xlwt 1.3.0 py37_0
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
yapf 0.30.0 py_0
zeromq 4.3.3 he6710b0_3
zict 2.0.0 py_0
zipp 3.4.0 pyhd3eb1b0_0
zlib 1.2.11 h7b6447c_3
zope 1.0 py37_1
zope.event 4.5.0 py37_0
zope.interface 5.1.2 py37h7b6447c_0
zstd 1.4.5 h9ceee32_0
Thanks!
Jun
Dear Jun,
The issue seems to be that you are trying to write more than 500 permutations to file. That doesn't work unfortunately due to an HDF5 error.