shishenyxx/DeepMosaic

Errors at multiprocessing and matplotlib

Closed this issue · 3 comments

Hi, Dr. Yang. I am trying to test DeepMosaic with files in demo after launching, it shows error below:

$ deepmosaic-draw.py -i /hdd1/DeepMosaic/demo/input.txt -o ./ -a /hdd1/annovar/

NOTICE: Output files are written to ./input.variant_function, ./input.exonic_variant_function
NOTICE: Reading gene annotation from /hdd1/annovar/humandb/hg19_refGene.txt ... Done with 72567 transcripts (including 17617 without coding sequence annotation) for 28263 unique genes
NOTICE: Processing next batch with 4 unique variants in 4 input lines
NOTICE: Output file with variants matching filtering criteria is written to ./input.hg19_gnomad_genome_dropped, and output file with other variants is written to ./input.hg19_gnomad_genome_filtered
NOTICE: Processing next batch with 4 unique variants in 4 input lines
NOTICE: Database index loaded. Total number of bins is 28127612 and the number of bins to be scanned is 4
NOTICE: Scanning filter database /hdd1/annovar/humandb/hg19_gnomad_genome.txt...Done

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "/hdd1/DeepMosaic/deepmosaic/featureExtraction.py", line 120, in multiprocess_iterator
fig1.savefig(image_file)
File "/usr/lib/python3/dist-packages/matplotlib/figure.py", line 2180, in savefig
self.canvas.print_figure(fname, **kwargs)
File "/usr/lib/python3/dist-packages/matplotlib/backend_bases.py", line 2021, in print_figure
canvas = self._get_output_canvas(format)
File "/usr/lib/python3/dist-packages/matplotlib/backend_bases.py", line 1961, in _get_output_canvas
raise ValueError(
ValueError: Format 'jpg' is not supported (supported formats: eps, pdf, pgf, png, ps, raw, rgba, svg, svgz)
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/hdd1/DeepMosaic/deepmosaic/./deepmosaic-draw.py", line 4, in
if name=='main': main()
File "/hdd1/DeepMosaic/deepmosaic/featureExtraction.py", line 230, in main
results = pool.map(multiprocess_iterator, all_variants, 8)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
raise self._value
ValueError: Format 'jpg' is not supported (supported formats: eps, pdf, pgf, png, ps, raw, rgba, svg, svgz)

The issue matplotlib seems to be resolved by re-installing pillow==6.0 (my pillow==7.0 and matplotlib==3.1.2.)
Could you help me resolving multiprocessing issues? I could see some other StackOverFlow or boards for similar thing, but there is no clear resolution.
Thank you!

Hi khy900712,

Thank you for your question, although it seems to be a systematic issue, I can provide some suggestions (not sure it will work). I noticed that you are using Python 3.8. Maybe you can test it with Python 3.7. We are working to provide some docker images to make version control easier.

Best,

Xiaoxu

Thanks, Xiaoxu. I will try it in Python 3.7. I am thankful if you offer some docker image. Looking forward to seeing it.
Best wishes,
Hyunyong (Howard)

Thanks, Xiaoxu. I will try it in Python 3.7. I am thankful if you offer some docker image. Looking forward to seeing it. Best wishes, Hyunyong (Howard)

Hi Hyunyong,

As most of the HPCs don't allow admin privilege, we made a singularity image. The singularity image (https://cloud.sylabs.io/library/arzoopatel5/deepmosaic/deepmosaic) is published today and DeepMosaic is updated to 1.0.2. We also updated the documentation accordingly.

Best,

Xiaoxu