Database installation problems
agudeloromero opened this issue · 4 comments
Hi there,
Good job for this awesome pipeline! I'd really like to try it, but I'm stuck on installing the databases.
I'm working on a cluster and tried:
hecatomb install --profile slurm
hecatomb install
Please see the log files enclosed.
Thanks in advance,
Patricia
2022-02-03T082640.845628.snakemake.log
2022-02-03T082606.247106.snakemake.log
Hi Patricia,
Thanks for your interest in the pipeline!
There's not much to go on here, but it looks like you have successfully downloaded some of the files in a previous run?
It looks like Snakemake had issues submitting the slurm jobs with the profile.
Do you have any slurm logs? (if you're using the example profile then they will be in logs/, otherwise they'll just be slurm-12345...log etc if using the cookiecutter profile). You might need to adjust the profile for your cluster. You should be able to access most of the information you need with sinfo -Nl and of course you clusters user guide.
Hi Michael,
Thanks for your quick reply.
I fixed this issue changing the database path out of conda in my config file.
Cheers,
Excellent news!
@beardymcjohnface I have an issue downloading the databases for HPC:
Looks like my install worked but I can't download the databases for HPC:
(/lustre/project/taw/share/conda-envs/hecatomb) [kvigil@cypress2 conda-envs]$ hecatomb --help
██╗ ██╗███████╗ ██████╗ █████╗ ████████╗ ██████╗ ███╗ ███╗██████╗
██║ ██║██╔════╝██╔════╝██╔══██╗╚══██╔══╝██╔═══██╗████╗ ████║██╔══██╗
███████║█████╗ ██║ ███████║ ██║ ██║ ██║██╔████╔██║██████╔╝
██╔══██║██╔══╝ ██║ ██╔══██║ ██║ ██║ ██║██║╚██╔╝██║██╔══██╗
██║ ██║███████╗╚██████╗██║ ██║ ██║ ╚██████╔╝██║ ╚═╝ ██║██████╔╝
╚═╝ ╚═╝╚══════╝ ╚═════╝╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝╚═════╝
Hecatomb version v1.1.0
Usage: hecatomb [OPTIONS] COMMAND [ARGS]...
Subcommand usage: hecatomb subcommand --help
Options:
--help Show this message and exit.
Commands:
run Run Hecatomb
test Run the Hecatomb test dataset
config Copy the system default config file
install Install the Hecatomb databases
combine Combine multiple Hecatomb runs
add-host Add a new host genome to use with Hecatomb
list-hosts List the available host genomes
(/lustre/project/taw/share/conda-envs/hecatomb) [kvigil@cypress2 conda-envs]$
(/lustre/project/taw/share/conda-envs/hecatomb) [kvigil@cypress2 conda-envs]$ hecatomb install --profile slurm
██╗ ██╗███████╗ ██████╗ █████╗ ████████╗ ██████╗ ███╗ ███╗██████╗
██║ ██║██╔════╝██╔════╝██╔══██╗╚══██╔══╝██╔═══██╗████╗ ████║██╔══██╗
███████║█████╗ ██║ ███████║ ██║ ██║ ██║██╔████╔██║██████╔╝
██╔══██║██╔══╝ ██║ ██╔══██║ ██║ ██║ ██║██║╚██╔╝██║██╔══██╗
██║ ██║███████╗╚██████╗██║ ██║ ██║ ╚██████╔╝██║ ╚═╝ ██║██████╔╝
╚═╝ ╚═╝╚══════╝ ╚═════╝╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝╚═════╝
Hecatomb version v1.1.0
[2023:01:25 08:33:58] Config file hecatomb.config.yaml already exists. Using existing config file.
[2023:01:25 08:33:58] Writing runtime config file to hecatomb.config.yaml
[2023:01:25 08:33:58] ------------------
[2023:01:25 08:33:58] | Runtime config |
[2023:01:25 08:33:58] ------------------
BigJobCpu: 24
BigJobMem: 64000
BigJobTimeMin: 1440
COMPRESSION: 1
CONTIG_MINLENGTH: 1000
CUTTAIL_WINDOW: 25
DEDUP_ACCURACY: 4
Databases: null
ENTROPY: 0.5
ENTROPYWINDOW: 25
Host: human
MediumJobCpu: 16
MediumJobMem: 32000
MoreRamCpu: 2
MoreRamMem: 16000
Output: hecatomb.out
Preprocess: paired
QSCORE: 15
READ_MINLENGTH: 90
Reads: /lustre/project/taw/share/conda-envs/hecatomb/bin/../test_data
Search: sensitive
SmallJobCpu: 8
SmallJobMem: 16000
canuSettings: correctedErrorRate=0.16 maxInputCoverage=10000 minInputCoverage=0 corOutCoverage=10000
corMhapSensitivity=high corMinCoverage=0 useGrid=False stopOnLowCoverage=False genomeSize=10M
-nanopore
filtAAprimary: --min-length 30 -e 1e-3
filtAAsecondary: --min-length 30 -e 1e-5
filtNTprimary: --min-length 90 -e 1e-10
filtNTsecondary: --min-length 90 -e 1e-20
linclustParams: --kmer-per-seq-scale 0.3 -c 0.8 --cov-mode 1 --min-seq-id 0.97 --alignment-mode
3
perfAA: --start-sens 1 --sens-steps 3 -s 7 --lca-mode 2 --shuffle 0
perfAAfast: -s 4.0 --lca-mode 2 --shuffle 0
perfNT: --start-sens 2 -s 7 --sens-steps 3
perfNTfast: -s 4.0
taxIdIgnore: 0 1 2 10239 131567 12429 2759
[2023:01:25 08:33:58] ---------------------
[2023:01:25 08:33:58] | Snakemake command |
[2023:01:25 08:33:58] ---------------------
snakemake -s /lustre/project/taw/share/conda-envs/hecatomb/bin/../snakemake/workflow/DownloadDB.smk --configfile hecatomb.config.yaml --rerun-incomplete --printshellcmds --nolock --show-failed-logs --profile slurm
Error: profile given but no config.yaml found. Profile has to be given as either absolute path, relative path or name of a directory available in either /etc/xdg/snakemake or /home/kvigil/.config/snakemake.
[2023:01:25 08:34:08] Error: Snakemake failed