AlexTISYoung/snipar

AttributeError: 'NoneType' object has no attribute 'dumps' when running impute_runner.py

yystat opened this issue · 11 comments

yystat commented

Dear Prof.,

Sorry to bother you again. When I'm running the following example codes (from https://sibreg.readthedocs.io/en/master/tutorial.html)

python impute_runner.py 1 2 test_data/sample.segments.gz test_data/sample1 --king test_data/sample.king --agesex test_data/sample.agesex --out_prefix test_data/sample --threads 4

It could successfully impute chromosome 1. However, after written the result for chromosome 1, an error occurred and the process stopped. The error message is:

Traceback (most recent call last):
  File "/s/pkg/linux64/python/3.8.3/lib/python3.8/multiprocessing/pool.py", line 268, in __del__
  File "/s/pkg/linux64/python/3.8.3/lib/python3.8/multiprocessing/queues.py", line 362, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

I barely know python and I don't know what causes this problem. Does it relate to the codes on line 195 for chromosome in range(args.from_chr, args.to_chr) and line 199 consumed_time = pool.map(run_imputation, inputs) in impute_runner.py? Could you please help me with this?

Of course, the above codes are supposed to impute chromosome 1 only and an error message after the job's done may not really matter. I also tried change 1 2 to 1 3 and it could impute chrs 1 and 2, and after than same error message.

Thank you very much!

Josh

AlexTISYoung commented
MoeenNehzati commented

Hi Josh,
I looked the error up and it's probably due to some inconsistencies between installed packages. Have you installed it on a fresh virtual environment? This way you won't experience any problem related to other installed packages. Can you give installing on a fresh virtual environment a try(if you haven't already)? Aside from that, could you provide me with the output of "pip freeze" command on your computer?
Thanks!
Moeen

yystat commented

Thank you Moeen!

I'm using my department's server and I don't think I could install it on a virtual environment. The IT person installed Python3.8 for me. Below are the outputs of the pip freeze command:

Babel==0.9.6
backports-abc==0.5
backports.ssl-match-hostname==3.5.0.1
cffi==1.6.0
chardet==2.2.1
configobj==4.7.2
configshell-fb==1.1.25
cryptography==1.7.2
cupshelpers==1.0
decorator==3.4.0
dnspython==1.12.0
docopt==0.6.2
ebs==0.0.10
enum34==1.0.4
ethtool==0.8
fontforge==1.0
futures==3.1.1
gssapi==1.2.0
idna==2.4
iniparse==0.4
ipaclient==4.6.6
ipaddress==1.0.16
ipalib==4.6.6
ipaplatform==4.6.6
ipapython==4.6.6
IPy==0.75
javapackages==1.0.0
Jinja2==2.7.2
jwcrypto==0.4.2
kitchen==1.1.1
kmod==0.1
langtable==0.0.31
lxml==3.2.1
MarkupSafe==0.11
msgpack==0.6.2
natsort==5.4.1
netaddr==0.7.5
netifaces==0.10.4
perf==0.1
plinkio==0.9.7
ply==3.4
policycoreutils-default-encoding==0.1
psutil==5.6.7
py-rrdtool==0.2.2
pyasn1==0.1.9
pyasn1-modules==0.0.8
pycparser==2.14
pycrypto==2.6.1
pycups==1.9.63
pycurl==7.19.0
pyfaidx==0.5.5
pygobject==3.22.0
pygpgme==0.3
pyliblzma==0.5.3
pyparsing==1.5.6
pysmbc==1.0.13
python-augeas==0.5.0
python-dateutil==1.5
python-ldap==2.4.15
python-linux-procfs==0.4.9
python-nss==0.16.0
python-yubico==1.2.3
pyudev==0.15
pyusb==1.0.0b1
pyxattr==0.5.1
PyYAML==3.11
pyzmq==15.3.0
qrcode==5.0.1
requests==2.6.0
rtslib-fb==2.1.69
salt==3000.3
schedutils==0.4
seobject==0.1
sepolicy==1.1
singledispatch==3.4.0.3
six==1.9.0
slip==0.4.0
slip.dbus==0.4.0
snpflip==0.0.6
SSSDConfig==1.16.4
targetcli-fb===2.1.fb49
tornado==4.2.1
urlgrabber==3.10
urllib3==1.10.2
urwid==1.1.1
yum-langpacks==0.4.2
yum-metadata-parser==1.1.4
MoeenNehzati commented

We have tested the package on python 3.6 so that might also be an issue. Am I safe to assume you can install new packages?

yystat commented

Yes, I think so.

MoeenNehzati commented

Ok. It means you should be able to create virtual environments. You just need to install venv first(via pip install venv).
Create a clean virtual environment, preferably a python3.6 environment, using venv, and then use that for working with the package.
python3.6 -m venv name-of-venv-directory should give you a python3.6 environment. After this whenever you enter source name-of-venv-directory/bin/activate the default python your terminal uses will become your virtual environment.

yystat commented

Thanks. It seems that I could not install it.
If 3.6 works, then I could ask the IT person to install 3.6 or even 3.7 for me.

yystat commented

Our server has python3.6.5, and the installation has errors when I run python setup.py pytest (in total there are 8 failed, 6 passed, 3 warnings):

subprocess.CalledProcessError: Command '['python3.6', 'impute_runner.py', '1', '3', 'test_data/sample.segments.gz', 'test_data/sample~', '--king', 'test_data/sample.king', '--agesex', 'test_data/sample.agesex', '--out_prefix', 'outputs/tmp/test_sample_imputed']' returned non-zero exit status 1.

ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /misc/s/pkg/linux64/python/3.6.5/lib/python3.6/site-packages/pysnptools/snpreader/wrap_plink_parser.cpython-36m-x86_64-linux-gnu.so)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "impute_runner.py", line 199, in <module>
    consumed_time = pool.map(run_imputation, inputs)
  File "/s/pkg/linux64/python/3.6.5/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/s/pkg/linux64/python/3.6.5/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /misc/s/pkg/linux64/python/3.6.5/lib/python3.6/site-packages/pysnptools/snpreader/wrap_plink_parser.cpython-36m-x86_64-linux-gnu.so)

Then I ran bash tests/generate_test_population.sh and got the following error message which should be due to the same reason:

Traceback (most recent call last):
  File "example/simulate_trait_quad.py", line 33, in <module>
    gts = gts_f.read().val
  File "/s/pkg/linux64/python/3.6.5/lib/python3.6/site-packages/pysnptools/snpreader/snpreader.py", line 460, in read
    val = self._read(None, None, order, dtype, force_python_only, view_ok)
  File "/s/pkg/linux64/python/3.6.5/lib/python3.6/site-packages/pysnptools/snpreader/bed.py", line 244, in _read
    from pysnptools.snpreader import wrap_plink_parser
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /s/pkg/linux64/python/3.6.5/lib/python3.6/site-packages/pysnptools/snpreader/wrap_plink_parser.cpython-36m-x86_64-linux-gnu.so)
MoeenNehzati commented

Apparently you are having problems with pysnptools(one of the dependencies). I'm not sure but I might be due to having an old version of libgcc. Can you show me the output of this command?
strings /lib64/libstdc++.so.6 | grep GLIBCXX

yystat commented

yes, below are the results:

GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_DEBUG_MESSAGE_LENGTH
MoeenNehzati commented

It's aligned with what I suspected. The dependencies need GLIBCXX_3.4.21 but you only have up to GLIBCXX_3.4.19.
This should be helpful. I know that you don't have admin access so maybe the solution with conda would solve your problem.