genomepy install error

[fgualdr]

Hi,

I am struggling to understand the issue but I cannot perform the post-processing step during genome installation.
Here is the error message:

Genome download successful, starting post processing...
Traceback (most recent call last):
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/bin/genomepy", line 10, in
sys.exit(cli())
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/genomepy/cli.py", line 221, in install
genomepy.install_genome(
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/genomepy/functions.py", line 305, in install_genome
p.download_genome(
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/genomepy/provider.py", line 303, in download_genome
shutil.move(src, dst)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/tempfile.py", line 827, in exit
self.cleanup()
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/tempfile.py", line 831, in cleanup
self._rmtree(self.name)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/tempfile.py", line 813, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/shutil.py", line 719, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/shutil.py", line 717, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/hpcnfs/home/ieo5244/.local/share/genomes/hg38/tmp384o4j6u'

[siebrenf]

This is very odd! The genome is moved from a temporary directory to the genomes directory with shutil.move(src, dst), and I'm not really sure how that is causing this error.

Thank you for providing the traceback. I'll let you know ASAP.

[siebrenf]

Using genomepy 0.9.3, running genomepy install hg38 -p ucsc, I cannot recreate this bug.

My best guess is it is related to a bug in tempfile. You might be able to circumvent it with these steps:

• open /hpcnfs/data/GN2/fgualdrini/.conda/envs/gimme/lib/python3.8/site-packages/genomepy/provider.py
• add rm_rf(infa) after line 267 keys_in = Fasta(infa).keys()

Please let me know if it works!

[siebrenf]

alternatively, you can try to change shutil.move(src, dst) to os.rename(src, dst)

[fgualdr]

Thanks,

so the genome-you I have was installed in the condo environment with gimme motif.
the version is:
genomepy, version 0.9.1

at line 261-276 in the provider.py I have the following:
# unzip genome
if link.endswith(".tar.gz"):
tar_to_bigfile(fname, fname)
elif link.endswith(".gz"):
os.rename(fname, fname + ".gz")
ret = sp.check_call(["gunzip", "-f", fname])
if ret != 0:
raise Exception(f"Error gunzipping genome {fname}")

        def regex_filer(_fname, _regex, _v):
            os.rename(_fname, _fname + "_to_regex")
            infa = _fname + "_to_regex"
            outfa = _fname
            filter_fasta(infa, outfa, regex=_regex, v=_v, force=True)

            return [k for k in Fasta(infa).keys() if k not in Fasta(outfa).keys()]

[siebrenf]

Then it might be fixed by updating genomepy!
Let me know if the error persists with version 0.9.3.

[siebrenf]

Feel free to reopen the issue if the problem wasn't fixed!