taskset fails to set pid XXX's affinity

[andreaminio]

Hi Ruibang,

I'm trying to use the tool on grape. I was able to set it to work on our department's cluster but for some instances it happens that I get this error when running callVarBam.py:

Delay 9 seconds before starting variant calling ...
taskset: failed to set pid XXXX's affinity: Invalid argument
callVar.py exited with exceptions. Exiting...
samtools view: writing to standard output failed: Broken pipe
samtools view: error closing standard output: -1
samtools view: writing to standard output failed: Broken pipe

It happens when I have multiple processes running on the same machine (I give 8 cores/4Gb of ram for each chromosome analysis), when I have more than 2 processes running together at least one dies with this message.

Looking at the code, I see that a taskset is call using the variabel set at line 101:

    cpuSet = ",".join(str(x) for x in random.sample(xrange(0, maxCpus), numCpus))
    taskSet = "taskset -c %s" % cpuSet

this makes me guess that the random selection of the cores (8 out of 80 in the machine) may be the origin of the issue, as more instances are run on the same machine the probability of selecting a second time a cpu already in use increases.

Is there any workaround to this issue?

Andrea

[aquaskyline]

I'm not able to repeat the problem on my side. afaik, taskset allows a core to be assigned more than once. A quick workaround might be removing taskset by changing
taskSet = "taskset -c %s" % cpuSet
to
taskSet = "%s" % cpuSet