Capture our next steps
andreww opened this issue · 6 comments
How do we take this forward?
A few thoughts:
- Testing needs to be more extensive.
- As does error handling.
- Getting it into pypi would help make it available to more people.
- Packaging into Debian/Redhat/Arch package feeds (and possibly rewriting in a compiled language) could allow this to become a standard tool available on any linux system.
- Integration with cron and/or slurm would create a whole set of additional ways to use this package. Perhaps these would be good targets for future hackathons.
- We need to get some content about this into the Turing Way. The book dash will be a good first opportunity for that.
- There was a discussion session around this topic which had a number of us in. This will end up producing a blog post for the SSI's blog. Do we try and mention this in that blog post? or do we see if we can do another blog post (usually they are happy to get posts) or two about this?
- Stay on the slack channel from cw23, we can continue to use that for talking.
- Do we want to meetup online to discuss this a bit more?
There is now a meeting poll on the CW23 slack for 9.
I have done the poll and hope to be able to discuss this with you all there soon. Exciting times!
To add to Colin's thoughts, all of which I agree with and seem very wise, a brain dump of a few of my own, to pre-register before we meet again:
-
I think
catsis distinct, significant and will eventually be (but is already a good way along to being) mature enough to submit as a piece of software to a journal such as JOSS if we can ensure it is justifiable as not something that could be classified under "minor 'utility' packages, including 'thin' API clients" which "are not acceptable" for submission, according to their scope (quoted from 'Scope & submission requirements' under the page linked in the previous sentence). I think it is far more than that personally, but some refinement might help to make that case strong!What's nice about JOSS is that their review criteria form a checklist of most if not all of the good things that an open package should have, so by submitting in this way, or even if not submitting, we can be guided by their open review criteria/checklist. We can probably tick many if not most items off already?
-
(A notable omission from the checklist above, but regardless...) We should set up some basic documentation, namely more than just the README page, but still something lightweight, mostly to provide an API reference for the command-line options. Sphinx is a very popular tool for easy and powerful docs set-up, and I am happy to volunteer to get the basic pages up in Sphinx, assuming folk are happy with that choice. I'll try to raise it in the meeting.
-
Colin said above:
Integration with cron and/or slurm would create a whole set of additional ways to use this package. Perhaps these would be good targets for future hackathons.
and in general I think a key part of development for future should be towards making
catsmore configurable, so, as well as supporting integration with various other schedulers in the way Colin suggests, it supports (by specification against the default), e.g:- other algorithms for estimating the more carbon-efficient time to run based on the carbon intensity API data information;
- deadlines for scheduling that are under the next 48 hours (e.g. schedule the task at the lowest carbon intensity within a given t < 48 hours (because often people might not want to wait that long) if that is possible?
I like the JOSS idea!
I love the JOSS idea.
The one other thing I would add is to try to have a clear 'user tool' and 'API for other use' distinction within cats. This ought to give a clear way to integrate with other schedulers (and I realise that there will be some systems that do their own scheduling, so a good way to find out when to run something could be useful for that case).
I think this thread can be closed as the tasks identified now have their own issues or have been completed.