Questions about FIRE installation
Closed this issue · 9 comments
I installed FIRE following the instructions and downloaded the testing files. I have tried running it from an interactive session in a cluster. The command was run from the FIRE directory, and I created a profile file to specify the needs of the server I was working with.
Command:
snakemake --configfile ../../fire_test/config.yaml --profile ./profiles/midway3 --local-cores 8 -k
Error:
Using profiles ./profiles/midway3 and workflow/profiles/default and workflow specific profile workflow/profiles/default for setting default command line arguments.
usage: snakemake [-h] [--dry-run] [--profile PROFILE] [--workflow-profile WORKFLOW_PROFILE] [--cache [RULE ...]] [--snakefile FILE] [--cores N] [--jobs N]
[--local-cores N] [--resources NAME=INT [NAME=INT ...]] [--set-threads RULE=THREADS [RULE=THREADS ...]] [--max-threads MAX_THREADS]
[--set-resources RULE:RESOURCE=VALUE [RULE:RESOURCE=VALUE ...]] [--set-scatter NAME=SCATTERITEMS [NAME=SCATTERITEMS ...]]
...
snakemake: error: unrecognized arguments: --cluster-cancel=scancel --cluster-cancel-nargs=500 --cluster-status=profiles/midway3/status-sacct.sh --reason=True
The config file midway3 is the same as the one in checkpoint, only adding the account and partition needed to run the test. But in general it seems like my profile is not being appropriately read or it has some error.
Can you suggest some pointers on how to fix this?
Snakemake v8 very recently made many code-breaking changes; can you please try installing snakemake v7.32.
Thank you for your help, I tried using the snakemake version you mentioned. However, in both attempts, with the versions 8 and 7, I get an error in a couple of lines: $PATH/FIRE/workflow/Snakefile:38: SyntaxWarning: invalid escape sequence '\s' data = pd.read_csv(config["manifest"], sep="\s+", comment="#").set_index("sample") that seem to be solved by adding an r, like: sep=r"\s+".
After that, the definition of the excludes variable is also called in question, it seems that under some conditions, it can be defined either as list or as character, which do not allow the concatenation. (Lines 16)
I solved that by forcing the variable to be a string vector. However, when trying to run the workflow over the testing data you suggest, the error starts like this: FileNotFoundError in file $PATH/FIRE/workflow/Snakefile, line 41: [Errno 2] No such file or directory: ' $PATH/fire_test/chr20.fa .fai ' It seems there are 2 spaces that are added there that might be not directing the pipeline to the correct file. $PATH is just a way to express the remainder of my path.
I have been trying to hunt down an issue like this with k-mer-variant-pipeline with Adriana when she runs that pipeline. A space is added when creating the index file name. I realize it's not FIRE, but it seems like the same error. There is this warning in the k-mer-variant-pipeline before the error with the fasta index occurs.
k-mer-variant-phasing/workflow/Snakefile:50: SyntaxWarning: invalid escape sequence '\d'
scatteritem="\d+-of-\d+",
The versions show: Running Snakemake version 7.32.4 and python version 3.12.1
The change log for python 3.12 shows a change to regular expressions. I'm unsure if it's related to the fasta file index-space problem as yet.
A backslash-character pair that is not a valid escape sequence now generates a SyntaxWarning, instead of DeprecationWarning. For example, re.compile("\d+.\d+") now emits a SyntaxWarning ("\d" is an invalid escape sequence, use raw strings for regular expression: re.compile(r"\d+.\d+")). In a future Python version, SyntaxError will eventually be raised, instead of SyntaxWarning. (Contributed by Victor Stinner in gh-98401.)
The file adding a space issue is a bug in snakemake caused by a change in f-strings in python 3.12. The workaround is to use a python version less than that. See this thread for lots of details:
snakemake/snakemake#2480
I think I shared with @adrisede previously but maybe not. The correct and coming solution here is to upgrade to snakemake v8. But that requires some code changes I haven't completed and then testing.
The other is just a warning that does not actually impact the run, but we should fix by prefixing r as suggested here.
Sounds good to me - Adriana downgraded to python 3.11 and verified that things started running correctly.
I will follow your lead then, thank you for the quick response