Executing bashrc

[IanSudbery]

I have noticed that one of the recent updates means that the users ~/.bashrc file is executed when a job is submitted to the cluster.

This is a problem for me because my ~/.bashrc does all sorts of things with my conda environment, the upshot of which is that my default environment, rather than the cgat-core conda environment is active when the statement is executed.

Now I could go in and refudge this. But its a bit of a hassle to have to rewrite my bashrc everything I decide to use a different conda env. And its becomes a problem when I need to use one env in one terminal and a different env in a different terminal

[sebastian-luna-valero]

Could you point me to the offending line of code, please?

[IanSudbery]

I don't know for sure, I just now that it is executing .bashrc now, when it didn't used to, but I'm guessing its Pipeline.Execution.execute:198:

 os.environ.update({'BASH_ENV': os.path.join(os.environ['HOME'], '.bashrc')})

which has been added since I last pulled (which might have been quite some time ago).

[IanSudbery]

Looking more closely, I now think that this is particular to Pipeline.Database.load, which sets to_cluster=False, and thus the above line gets executed, whereas when a job is submitted to the cluster, the local environment is passed via -V, rather than by executing bashrc.

[sebastian-luna-valero]

That line was added in #16

Please see discussion in CGATOxford/CGATPipelines#411

Moreover, to_cluster=False is required for Pipeline.Database.load to avoid sqlite operational errors.

Looking at the code, in Pipeline.Execution we have:

    os.environ.update({'BASH_ENV': os.path.join(os.environ['HOME'], '.bashrc')})
    process = subprocess.Popen(statement % kwargs,
                               cwd=cwd,
                               shell=True,
                               stdin=subprocess.PIPE,
                               stdout=subprocess.PIPE,
                               stderr=subprocess.PIPE,
                               env=os.environ.copy(),
                               executable="/bin/bash")

Where as in Pipeline.Cluster we have:

jt.jobEnvironment = {'BASH_ENV': '~/.bashrc'}

In addition to -V which presumably are equivalent?

Does getting rid of os.environ.update({'BASH_ENV': os.path.join(os.environ['HOME'], '.bashrc')}) solve your issues?

[Acribbs]

This is quite an old issue, if there are still problems feel free to re-open