Running ConFindr with weird internet access
Closed this issue · 2 comments
lskatz commented
Hi, I have some cgMLST databases and have formatted one of them with confindr but it still tries to download other databases when it initializes. I am behind a funky firewall and so I want to try to run this without internet access. How can I tell it to just look at one database and not try to download others? Or is this sort of the wrong way to think about things? Thank you for any help.
2022-05-24 18:34:11 Welcome to ConFindr 0.7.4! Beginning analysis of your samples...
2022-05-24 18:34:11 Could not find Escherichia_db_cgderived.fasta
2022-05-24 18:34:11 Could not find Listeria_db_cgderived.fasta
2022-05-24 18:34:11 Could not find Salmonella_db_cgderived.fasta
2022-05-24 18:34:11 Could not find refseq.msh
2022-05-24 18:34:11 Databases not present - downloading basic databases now...
2022-05-24 18:34:11 Downloading mash refseq sketch...
2022-05-24 18:34:11 Downloading cgMLST-derived data for Escherichia, Salmonella, and Listeria...
Traceback (most recent call last):
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 1350, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1281, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1327, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1276, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1036, in _send_output
self.send(msg)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 976, in send
self.connect()
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1451, in connect
server_hostname=server_hostname)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/ssl.py", line 423, in wrap_socket
session=session
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/ssl.py", line 870, in _create
self.do_handshake()
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/bin/confindr.py", line 10, in <module>
sys.exit(main())
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/confindr.py", line 1214, in main
confindr(args)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/confindr.py", line 1031, in confindr
check_for_databases_and_download(database_location=args.databases)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/confindr.py", line 930, in check_for_databases_and_download
download_cgmlst_derived_data(database_location)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/database_setup.py", line 235, in download_cgmlst_derived_data
os.path.join(output_folder, 'confindr_db.tar.gz'))
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 1393, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 1352, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091)>
lskatz commented
Or also I tried with 7-gene MLST to just get started and it doesn't seem to work there either:
(confindr) [gzu2@monolith3 test_confindr]$ confindr_create_db -i senterica/ -o senterica.confindr -g Salmonella --desired_number_genes 7
# .... downloaded from NCBI okay ...
(confindr) [gzu2@monolith3 test_confindr]$ ls senterica
aroC.fasta dnaN.fasta.nhr hemD.fasta.nin hisD.fasta.nsq senterica.txt thrA.fasta
aroC.fasta.nhr dnaN.fasta.nin hemD.fasta.nsq purE.fasta sucA.fasta thrA.fasta.nhr
aroC.fasta.nin dnaN.fasta.nsq hisD.fasta purE.fasta.nhr sucA.fasta.nhr thrA.fasta.nin
aroC.fasta.nsq hemD.fasta hisD.fasta.nhr purE.fasta.nin sucA.fasta.nin thrA.fasta.nsq
dnaN.fasta hemD.fasta.nhr hisD.fasta.nin purE.fasta.nsq sucA.fasta.nsq
(confindr) [gzu2@monolith3 test_confindr]$ ls senterica.confindr -C | head
assembly_summary_refseq.txt genome_33.fasta.nin genome_670.fasta.nin
gene_hit_report.csv genome_33.fasta.nsq genome_670.fasta.nsq
genome_1000.fasta genome_340.fasta genome_671.fasta
genome_1000.fasta.nhr genome_340.fasta.nhr genome_671.fasta.nhr
genome_1000.fasta.nin genome_340.fasta.nin genome_671.fasta.nin
genome_1000.fasta.nsq genome_340.fasta.nsq genome_671.fasta.nsq
genome_1001.fasta genome_341.fasta genome_672.fasta
genome_1001.fasta.nhr genome_341.fasta.nhr genome_672.fasta.nhr
genome_1001.fasta.nin genome_341.fasta.nin genome_672.fasta.nin
genome_1001.fasta.nsq genome_341.fasta.nsq genome_672.fasta.nsq
(confindr) [gzu2@monolith3 test_confindr]$ confindr.py -i /scicomp/groups/OID/NCEZID/DFWED/EDLB/projects/validation/mlstComparison/illumina/weird-genomes.sneakernet -d senterica.confindr -o senterica.confindr.out --threads 12
2022-05-24 18:40:10 Welcome to ConFindr 0.7.4! Beginning analysis of your samples...
2022-05-24 18:40:10 Could not find Escherichia_db_cgderived.fasta
2022-05-24 18:40:10 Could not find Listeria_db_cgderived.fasta
2022-05-24 18:40:10 Could not find Salmonella_db_cgderived.fasta
2022-05-24 18:40:10 Databases not present - downloading basic databases now...
2022-05-24 18:40:10 Downloading mash refseq sketch...
2022-05-24 18:40:11 Downloading cgMLST-derived data for Escherichia, Salmonella, and Listeria...
Traceback (most recent call last):
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 1350, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1281, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1327, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1276, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1036, in _send_output
self.send(msg)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 976, in send
self.connect()
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/http/client.py", line 1451, in connect
server_hostname=server_hostname)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/ssl.py", line 423, in wrap_socket
session=session
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/ssl.py", line 870, in _create
self.do_handshake()
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/bin/confindr.py", line 10, in <module>
sys.exit(main())
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/confindr.py", line 1214, in main
confindr(args)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/confindr.py", line 1031, in confindr
check_for_databases_and_download(database_location=args.databases)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/confindr.py", line 930, in check_for_databases_and_download
download_cgmlst_derived_data(database_location)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/site-packages/confindr_src/database_setup.py", line 235, in download_cgmlst_derived_data
os.path.join(output_folder, 'confindr_db.tar.gz'))
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 247, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 1393, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/scicomp/home-pure/gzu2/bin/anaconda3/envs/confindr/lib/python3.7/urllib/request.py", line 1352, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091)>