urllib2.HTTPError: HTTP Error 404: Not Found
ABanaeiEsfahani opened this issue · 3 comments
Hello,
I had run cMonkey on my dataset while ago without any problem. But seems like I can't manage manage to run it again with slightly different parameters. Could you please help me? Thanks.
Best regards,
Amir
cmonkey2 ./Mtb/WayneAndTbX007_MedianNormalization_RatioByMeanD00.tsv --organism mtu --string ./Mtb/83332_v10.tsv --rsat_base_url http://pedagogix-tagc.univ-mrs.fr/rsat --rsat_organism Mycobacterium_tuberculosis_H37Rv_uid57777 --rsat_features gene --out ./Output_files --use_BSCM --numclusters 600
2016-11-15 16:14:58 INFO checking MEME...
2016-11-15 16:14:59 INFO Input matrix has # rows: 2683, # columns: 33
2016-11-15 16:14:59 INFO # clusters/row: 2
2016-11-15 16:14:59 INFO # clusters/column: 400
2016-11-15 16:14:59 INFO # CLUSTERS: 600
2016-11-15 16:14:59 INFO use operons: 1
2016-11-15 16:14:59 INFO using MEME version 4.10.2
2016-11-15 16:15:01 INFO attempting automatic download of operons from Microbes Online
2016-11-15 16:15:01 INFO Loading STRING file at './Mtb/83332_v10.tsv'
2016-11-15 16:15:01 INFO KEGG = 'Mycobacterium tuberculosis H37Rv uid57777' -> RSAT = 'Mycobacterium_tuberculosis_H37Rv_uid57777'
2016-11-15 16:15:01 INFO Creating networks...
2016-11-15 16:15:01 INFO stringdb.read_edges2()
2016-11-15 16:15:05 INFO Finished loading ./Mtb/83332_v10.tsv
2016-11-15 16:15:08 INFO Processing network 5%
2016-11-15 16:15:08 INFO Processing network 10%
2016-11-15 16:15:08 INFO Processing network 15%
2016-11-15 16:15:08 INFO Processing network 20%
2016-11-15 16:15:08 INFO Processing network 25%
2016-11-15 16:15:09 INFO Processing network 30%
2016-11-15 16:15:09 INFO Processing network 35%
2016-11-15 16:15:09 INFO Processing network 40%
2016-11-15 16:15:09 INFO Processing network 45%
2016-11-15 16:15:09 INFO Processing network 50%
2016-11-15 16:15:10 INFO Processing network 55%
2016-11-15 16:15:10 INFO Processing network 60%
2016-11-15 16:15:10 INFO Processing network 65%
2016-11-15 16:15:10 INFO Processing network 70%
2016-11-15 16:15:10 INFO Processing network 75%
2016-11-15 16:15:11 INFO Processing network 80%
2016-11-15 16:15:11 INFO Processing network 85%
2016-11-15 16:15:11 INFO Processing network 90%
2016-11-15 16:15:11 INFO Processing network 95%
2016-11-15 16:15:11 INFO Processing network 100%
2016-11-15 16:15:11 WARNING 15 (out of 1247720) nodes not found in synonyms
2016-11-15 16:15:11 WARNING 1292 (out of 1247720) nodes not found in canonical gene names
2016-11-15 16:15:12 INFO stringdb.read_edges2(), 407198 edges read, 216662 edges ignored
2016-11-15 16:15:14 INFO MicrobesOnline - make_network()
2016-11-15 16:15:14 INFO MicrobesOnline.get_operon_predictions_for(83332)
Traceback (most recent call last):
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/bin/cmonkey2", line 36, in
cmonkey_run.run()
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/cmonkey_run.py", line 512, in run
self.prepare_run()
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/cmonkey_run.py", line 474, in prepare_run
thesaurus = self.organism().thesaurus()
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/cmonkey_run.py", line 231, in organism
self.__organism = self.make_organism()
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/cmonkey_run.py", line 341, in make_organism
self['fasta_file'])
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/organism.py", line 244, in init
fasta_file)
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/organism.py", line 117, in init
OrganismBase.init(self, code, network_factories, ratios=ratios)
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/organism.py", line 72, in init
self.__networks.append(make_network(self, ratios))
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/microbes_online.py", line 231, in make_network
edges = get_operon_edges(microbes_online, organism)
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/microbes_online.py", line 215, in get_operon_edges
__get_predictions(microbes_online, organism), organism)
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/microbes_online.py", line 199, in __get_predictions
organism.taxonomy_id())
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/microbes_online.py", line 54, in get_operon_predictions_for
return util.read_url_cached(url, cache_file).decode('utf-8')
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/util.py", line 255, in read_url_cached
outfile.write(read_url(url))
File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/lib/python2.7/site-packages/cmonkey/util.py", line 244, in read_url
return urlopen(url).read()
File "/cluster/apps/python/2.7.2/x86_64/lib64/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/cluster/apps/python/2.7.2/x86_64/lib64/python2.7/urllib2.py", line 400, in open
response = meth(req, response)
File "/cluster/apps/python/2.7.2/x86_64/lib64/python2.7/urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "/cluster/apps/python/2.7.2/x86_64/lib64/python2.7/urllib2.py", line 438, in error
return self._call_chain(*args)
File "/cluster/apps/python/2.7.2/x86_64/lib64/python2.7/urllib2.py", line 372, in _call_chain
result = func(*args)
File "/cluster/apps/python/2.7.2/x86_64/lib64/python2.7/urllib2.py", line 521, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found
Hi,
it looks like microbesonline.org has removed or moved the downloadable files for
operon predictions. In your case it would be a file called
http://www.microbesonline.org/operons/gnc83332.named
I have checked and at least right now those files are not available anymore.
This file is cached typically stored in the cache directory, so if you happen to
have this file from your previous runs, then you should be able to work around
this problem by copying it to your cache directory.
If you don't have that file anymore please let me know and I will send you that file.
In the meanwhile, we will contact microbesonline.org to find out what caused
this change and provide a workaround.
Thank you very much for bringing this to our attention.
MicrobesOnline fixed the issue and the download of operon predictions should work as before.
Thanks a lot :)