Create Package for pyglidein to be installed via pip
hskarlupka opened this issue · 7 comments
Package pyglidein so it can be installed via pip.
@dsschult I put all of the bash wrapper scripts into the pyglidein egg. I'm now trying to go through each submit class to determine a sane way to use the default wrapper scripts from the site-packages directory. It looks like each submit class tries to access and write the submit file a little differently. Here is what I'm thinking to clean this up:
- The default should be to not have a configuration for the glidein_start.sh script. If they user hasn't set any configuration variables, it is assumed they want to use the one packaged with pyglidein and it is on us to make sure the submit script is written properly and the file is in the correct location.
- Going forward there will only be two variables to define the glidein_start.sh script.
['Glidein']['glidein_script'] will contain the full path of the location of the glidein script. This will be used for things like transfer_inputs in condor or links in PBS.
['SubmitFile']['executble'] will contain the command to be run. This will go in the env_wrapper script for condor or the submit file in PBS. If this doesn't exist then os.path.basename(['Glidein']['glidein_script']) will be used.
I'd actually love to remove one of those two, since I think they were parallel developments. What is needed is a location to a modified glidein_start.sh script for testing a new cluster. That's all the input a user should have, in my opinion.
One thing I'd recommend is an option with a location for temporary submission files, like the env wrapper and submit file. If we install this on CVMFS, that's read-only, so it can't write to the install dir like it does now.
@dsschult I'll reduce it down to ['Glidein']['glidein_script']
I believe there are a bunch of options under ["SubmitFile"] that allow you to customized the directory for where your submit files land.
I haven't followed all the options that well. Do we have a place where they are all documented?
@dsschult @gonzalomerino @vbrik I created a Pypi account for myself. It looks like you can't create group accounts, but you can add users to projects. I'll need each of you to create accounts with Pypi.
I'm going to keep the project name pyglidein on Pypi. What should I use for an author email address? This will show up on the Pypi webage. For now I used author_email='contact-us@icecube.wisc.edu'