Atlas was not able to download a key database
Closed this issue · 2 comments
Hello,
I have tried to run atlas on the server a few times but every time I have the same problem, it seems to be unable to download a database. Are there any solutions?
So I started with "conda activate atlas-dev", then
"atlas init -w /dir --db-dir /dir",
and then
"atlas run all --working-dir /dir --config-file /dir/config.yaml --jobs 10".
So every time it left me with this:
log: logs/dram/download_dram.log (check log file(s) for error details)
conda-env: /dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_
shell:
DRAM-setup.py prepare_databases --output_dir /dir/databases/DRAM/db --threads 8 --verbose --skip_uniref &> logs/dram/download_dram.log ; DRAM-setup.py export_config --output_file /dir/databases/DRAM/DRAM.config
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Logfile logs/dram/download_dram.log:
/dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_/lib/python3.11/site-packages/mag_annotator/database_handler.py:169: UserWarning: Database does not exist at path None
warnings.warn(
/dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_/lib/python3.11/site-packages/mag_annotator/database_handler.py:126: UserWarning: Database does not exist at path None
warnings.warn("Database does not exist at path %s" % description_loc)
2024-10-09 02:26:11,995 - Starting the process of downloading data
2024-10-09 02:26:11,995 - Skipping UniRef
2024-10-09 02:26:11,995 - The kegg_loc argument was not used to specify a downloaded kegg file, and dram can not download it its self. So it is assumed that the user wants to set up DRAM without it
2024-10-09 02:26:11,995 - The gene_ko_link_loc argument was not used to specify a downloaded gene_ko_link file, and dram can not download it its self. So it is assumed that the user wants to set up DRAM without it
2024-10-09 02:26:11,995 - Database preparation started
2024-10-09 02:26:11,995 - Downloading kofam_hmm
2024-10-09 02:31:10,876 - Downloading kofam_ko_list
2024-10-09 02:31:44,463 - Downloading pfam
2024-10-09 02:40:28,769 - Downloading pfam_hmm
2024-10-09 02:40:29,582 - Downloading dbcan
2024-10-09 02:42:39,339 - Something went wrong with the download of the url: http://bcb.unl.edu/dbCAN2/download/dbCAN-HMMdb-V11.txt
2024-10-09 02:42:39,339 - <urlopen error [Errno 110] Connection timed out>
downloading ftp://ftp.genome.jp/pub/db/kofam/profiles.tar.gz
downloading ftp://ftp.genome.jp/pub/db/kofam/ko_list.gz
downloading ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.full.gz
downloading ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.dat.gz
downloading http://bcb.unl.edu/dbCAN2/download/dbCAN-HMMdb-V11.txt
Traceback (most recent call last):
File "/dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_/bin/DRAM-setup.py", line 187, in
args.func(**args_dict)
File "/dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_/lib/python3.11/site-packages/mag_annotator/database_processing.py", line 592, in prepare_databases
locs[i] = download_functions[i](
^^^^^^^^^^^^^^^^^^^^^^
File "/dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_/lib/python3.11/site-packages/mag_annotator/database_processing.py", line 169, in download_dbcan
download_file(link_path, dbcan_hmm, logger, verbose=verbose)
File "/dir/databases/conda_envs/c9d07354e296f0f3e7ea7066304ef57b_/lib/python3.11/site-packages/mag_annotator/utils.py", line 33, in download_file
raise URLError("DRAM whas not able to download a key database, check the logg for details")
urllib.error.URLError: <urlopen error DRAM whas not able to download a key database, check the logg for details>
Removing output files of failed job dram_download since they might be corrupted:
/dir/databases/DRAM/db
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Note the path to the log file for debugging.
Documentation is available at: https://metagenome-atlas.readthedocs.io
Issues can be raised at: https://github.com/metagenome-atlas/atlas/issues
Complete log: .snakemake/log/2024-10-08T142448.194252.snakemake.log
[Atlas] CRITICAL: Command 'snakemake --snakefile /dir/atlas/atlas/workflow/Snakefile --directory /dir --rerun-triggers mtime --jobs 10 --rerun-incomplete --configfile '/dir/config.yaml' --nolock --show-failed-logs --use-conda --conda-prefix /dir/databases/conda_envs --resources mem=1903 mem_mb=1949357 java_mem=1618 --scheduler greedy all ' returned non-zero exit status 1.
atlas, version 2.18.2+17.g22d78b98
I can't find any log files either, it says there is no such directory. Which is strange.
Thank you,
Best
Hey, we had problems with dram download before. In the meantime they should be solved. But the version 2.19 was not on bioconda. It should be in an hour or so.
Can you simply upodate atlas. mamba install metagenome-atlas=2.19
Hey, we had problems with dram download before. In the meantime they should be solved. But the version 2.19 was not on bioconda. It should be in an hour or so.
Can you simply upodate atlas.
mamba install metagenome-atlas=2.19
Hi,
I can't update it
conda install metagenome-atlas=2.19
/home/m/miniconda3/lib/python3.12/site-packages/conda/base/context.py:198: FutureWarning: Adding 'defaults' to channel list implicitly is deprecated and will be removed in 25.3.
To remove this warning, please choose a default channel explicitly with conda's regular configuration system, e.g. by adding 'defaults' to the list of channels:
conda config --add channels defaults
For more information see https://docs.conda.io/projects/conda/en/stable/user-guide/configuration/use-condarc.html
deprecated.topic(
/home/m/miniconda3/lib/python3.12/site-packages/conda/base/context.py:198: FutureWarning: Adding 'defaults' to channel list implicitly is deprecated and will be removed in 25.3.
To remove this warning, please choose a default channel explicitly with conda's regular configuration system, e.g. by adding 'defaults' to the list of channels:
conda config --add channels defaults
For more information see https://docs.conda.io/projects/conda/en/stable/user-guide/configuration/use-condarc.html
deprecated.topic(
Channels:
- defaults
- conda-forge
- bioconda
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- metagenome-atlas=2.19*
Current channels:
- defaults
- https://conda.anaconda.org/conda-forge/linux-64
- https://conda.anaconda.org/bioconda/linux-64
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.