watchexec/watchexec

watchexec produces no output with a fairly complex nextflow script

Closed this issue · 3 comments

ArchKudo commented

I am trying to run the command nextflow whenever a file changes in my current directory
Although it invokes a new execution on every file change, there is for some reason no output and the process seems stuck?
I tried running without watchexec and it runs without any issues
I also tried running a minimal script and it seems to work under watchexec, I can't seem to figure out why, any guidance would be useful

  • Watchexec's version

Have tried cargo-binstall and watchexec-cli from apt repo with similar results

base ❯ watchexec --version
watchexec 1.24.0 (477d59d 2023-12-09) +pid1
commit-hash: 477d59d319ba68a0e6eab9429130975d525e8687
commit-date: 2023-12-09
build-date: 2023-12-09
release: 1.24.0
features: default,pid1
  • The OS you're using
base ❯ uname -a
Linux wbar 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

  • A log with -vvv --log-file (if it has sensitive info you can email it at felix@passcod.name — do that after filing so you can reference the issue ID)

    • Simple process run
~/workplace/xyflo main* ⇡ 38s
base ❯ watchexec nextflow run tut.nf --resume
[Running: nextflow run tut.nf --resume]
N E X T F L O W  ~  version 23.04.3
Launching `tut.nf` [zen_swartz] DSL2 - revision: ef5b87854e
^C
[-        ] process > fe... -
~/workplace/xyflo main* ⇡ 3m 25s
base ❯ WARN: Killing running task
executor >  local (1)
[-        ] process > fe... -
  • Simple process run for minimal script
~/workplace/xyflo main* ⇡ 3m 25s
base ❯ watchexec nextflow run x.nf
[Running: nextflow run x.nf]
N E X T F L O W  ~  version 23.04.3
Launching `x.nf` [elated_gutenberg] DSL2 - revision: 84739fa49d
hello world
[Command was successful]
^C
  • Run without watchexec
~/workplace/xyflo main* ⇡ 8s
base ❯ nextflow run tut.nf --resume
N E X T F L O W  ~  version 23.04.3
Launching `tut.nf` [backstabbing_spence] DSL2 - revision: ef5b87854e
executor >  local (1)
[4b/21bc7e] process > fetchRunAccesio... [100%] 1 of 1 ✔
[ERR4334939, ERR4334940, ERR4334941, ERR4472601, ERR3224612, ERR3224613, ERR3224607, ERR3224614, ERR3224608, ERR3224609, ERR3224610, ERR3224611, ERR3275091, ERR3275092, ERR3275093, ERR3275094, ERR3275095, ERR3275096, ERR3275097, ERR3275098, ERR3275099, ERR3275100, ERR3275101, ERR3275102, ERR3275130, ERR3275103, ERR3275104, ERR3275105, ERR3275106, ERR3275107, ERR3275108, ERR3275131, ERR3275109, ERR3275110, ERR3275111, ERR3275132, ERR3275112, ERR3275133, ERR3275113, ERR3275114, ERR3275115, ERR3275116, ERR3275134, ERR3275135, ERR3275117, ERR3275118, ERR3275119, ERR3275136, ERR3275137, ERR3275120, ERR3275138, ERR3275121, ERR3275122, ERR3275123, ERR3275124, ERR3275125, ERR3275126, ERR3275127, ERR3275128, ERR3275129, ERR3224567, ERR3224568, ERR3224569, ERR3224570, ERR3224571, ERR3224572, ERR3224573, ERR3224574, ERR3224575, ERR3224576, ERR3224577, ERR3224578, ERR3224579, ERR3224580, ERR3224581, ERR3224582, ERR3224583, ERR3224584, ERR3224585, ERR3224586, ERR3224587, ERR3224588, ERR3224589, ERR3224590, ERR3224591, ERR3224592, ERR3224593, ERR3224594, ERR3224595, ERR3224596, ERR3224597, ERR3224598, ERR3224599, ERR3224600, ERR3224601, ERR3224602, ERR3224603, ERR3224604, ERR3224605, ERR3224606, ERR3211475, ERR3211476, ERR3211477, ERR3211478, ERR3211479, ERR3211480, ERR3211481, ERR3211482, ERR3211483, ERR3211484, ERR3211485, ERR3211486, ERR3220178, ERR3220179, ERR3220180, ERR3220181, ERR3220182, ERR3220183, ERR3220184, ERR3220185, ERR3220186, ERR3220187, ERR3220188, ERR3220189, ERR3220190, ERR3220191, ERR3220192, ERR3220193, ERR3220194, ERR3220195, ERR3220196, ERR3220197, ERR3220198, ERR3220199, ERR3220200, ERR3220201, ERR3220202, ERR3220203, ERR3220204, ERR3220205, ERR3220206, ERR3220207, ERR3220208, ERR3220209, ERR3220210, ERR3220211, ERR3220212, ERR3220213, ERR3220214, ERR3220215, ERR3220216, ERR3220225, ERR3220217, ERR3220218, ERR3220219, ERR3220220, ERR3220221, ERR3220222, ERR3220223, ERR3220224, ERR3211439, ERR3211440, ERR3211441, ERR3211442, ERR3211443, ERR3211444, ERR3211445, ERR3211446, ERR3211447, ERR3211448, ERR3211449, ERR3211450, ERR3211451, ERR3211452, ERR3211453, ERR3211454, ERR3211455, ERR3211456, ERR3211457, ERR3211458, ERR3211459, ERR3211460, ERR3211461, ERR3211462, ERR3211463, ERR3211464, ERR3211465, ERR3211466, ERR3211467, ERR3211468, ERR3211469, ERR3211470, ERR3211471, ERR3211472, ERR3211473, ERR3211474, ERR3201419, ERR3201420, ERR3201421, ERR3201422, ERR3201375, ERR3201376, ERR3201377, ERR3201378, ERR3201379, ERR3201380, ERR3201381, ERR3201382, ERR3201383, ERR3201384, ERR3201385, ERR3201386, ERR3201387, ERR3201388, ERR3201389, ERR3201390, ERR3201391, ERR3201392, ERR3201393, ERR3201394, ERR3201395, ERR3201396, ERR3201397, ERR3201398, ERR3201399, ERR3201400, ERR3201401, ERR3201402, ERR3201403, ERR3201404, ERR3201405, ERR3201406, ERR3201407, ERR3201408, ERR3201409, ERR3201410, ERR3201411, ERR3201412, ERR3201413, ERR3201414, ERR3201415, ERR3201416, ERR3201417, ERR3201418]
  • Verbose output

https://gist.github.com/ArchKudo/9d292607b597f76bca293307cc4b1ea4

  • A sample command that you've run that has the issue
watchexec nextflow run tut.nf --resume
 watchexec nextflow run x.nf

base ❯ cat x.nf
workflow {
        println "hello world"
}
base ❯ cat tut.nf
process fetchRunAccesionsForBioProject {                                                                                                      
    output:
        path 'readAccessions.txt'

    script:
    """
    wf.sh fetchRunAccesionsForBioProject "PRJEB31266" > "readAccessions.txt"
    """
}

workflow {
        i =      fetchRunAccesionsForBioProject()
        i.view { it.readLines() }
}

Thank you

passcod commented

Can you try running with 1.24.2

ArchKudo commented

Hi, thank you for replying unfortunately the issue persists

~/workplace/xyflo main* ⇡ 2m 52s
base ❯ watchexec --version
watchexec 1.24.2 (a1ce449 2023-12-20) +pid1
commit-hash: a1ce449fa408861dd877ea49cd6df6be3de2191b
commit-date: 2023-12-20
build-date: 2023-12-20
release: 1.24.2
features: default,pid1

~/workplace/xyflo main* ⇡
base ❯ cat tut.nf
println "hello world"
params.bioProjectId = "PRJEB31266"
process fetchRunAccesionsForBioProject {
    output:
        path 'readAccessions.txt'

    script:
    """
    wf.sh fetchRunAccesionsForBioProject "PRJEB31266" > "readAccessi
    """
}

workflow {
        i =      fetchRunAccesionsForBioProject()
        i.view { it.readLines() }
}


~/workplace/xyflo main* ⇡
base ❯ vim tut.nf

~/workplace/xyflo main* ⇡ 17s
base ❯ nextflow run tut.nf
N E X T F L O W  ~  version 23.04.3
Launching `tut.nf` [soggy_heisenberg] DSL2 - revision: ef5b87854e
executor >  local (1)
[db/db68c3] process > fetchRunAccesionsForBioProject [100%] 1 of 1 ✔
[ERR4334939, ERR4334940, ERR4334941, ERR4472601, ERR3224612, ERR32243224603, ERR3224604, ERR3224605, ERR3224606, ERR3211475, ERR3211476, ERR3211477, ERR3211478, ERR3211479, ERR3211480, ERR3211481, ERR3211482, ERR3211483, ERR3211484, ERR3211485, ERR3211486, ERR3220178, ERR3220179, ERR3220180, ERR3220181, ERR3220182, ERR3220183, ERR3220184, ERR3220185, ERR3220186, ERR3220187, ERR3220188, ERR3220189, ERR3220190, ERR3220191, ERR3220192, ERR3220193, ERR3220194, ERR3220195, ERR3220196, ERR3220197, ERR3220198, ERR3220199, ERR3220200, ERR3220201, ERR3220202, ERR3220203, ERR3220204, ERR3220205, ERR3220206, ERR3220207, ERR3220208, ERR3220209, ERR3220210, ERR3220211, ERR3220212, ERR3220213, ERR3220214, ERR3220215, ERR3220216, ERR3220225, ERR3220217, ERR3220218, ERR3220219, ERR3220220, ERR3220221, ERR3220222, ERR3220223, ERR3220224, ERR3211439, ERR3211440, ERR3211441, ERR3211442, ERR3211443, ERR3211444, ERR3211445, ERR3211446, ERR3211447, ERR3211448, ERR3211449, ERR3211450, ERR3211451, ERR3211452, ERR3211453, ERR3211454, ERR3211455, ERR3211456, ERR3211457, ERR3211458, ERR3211459, ERR3211460, ERR3211461, ERR3211462, ERR3211463, ERR3211464, ERR3211465, ERR3211466, ERR3211467, ERR3211468, ERR3211469, ERR3211470, ERR3211471, ERR3211472, ERR3211473, ERR3211474, ERR3201419, ERR3201420, ERR3201421, ERR3201422, ERR3201375, ERR3201376, ERR3201377, ERR3201378, ERR3201379, ERR3201380, ERR3201381, ERR3201382, ERR3201383, ERR3201384, ERR3201385, ERR3201386, ERR3201387, ERR3201388, ERR3201389, ERR3201390, ERR3201391, ERR3201392, ERR3201393, ERR3201394, ERR3201395, ERR3201396, ERR3201397, ERR3201398, ERR3201399, ERR3201400, ERR3201401, ERR3201402, ERR3201403, ERR3201404, ERR3201405, ERR3201406, ERR3201407, ERR3201408, ERR3201409, ERR3201410, ERR3201411, ERR3201412, ERR3201413, ERR3201414, ERR3201415, ERR3201416, ERR3201417, ERR3201418]


~/workplace/xyflo main* ⇡ 10s
base ❯ watchexec nextflow run tut.nf
[Running: nextflow run tut.nf]
N E X T F L O W  ~  version 23.04.3
Launching `tut.nf` [distracted_pasteur] DSL2 - revision: ef5b87854e
^C[Waiting 60s for processes to exit before stopping...]

~/workplace/xyflo main* ⇡ 4m 6s
ArchKudo commented

Was able workaround this with --no-process-group option, sorry for the trouble