Shovel is like Rake for python. Turn python functions into tasks simply, and access and invoke them from the command line. 'Nuff said. New Shovel also now has support for invoking the same tasks in the browser you'd normally run from the command line, without any modification to your shovel scripts.
- Tasks should be easy to define -- one decorator, no options
- Turning a function into task should change as little as possible -- we don't want you to have to change the function's interface at all
- Arguments are strings -- rather than guess about the type of input parameters, we're just going to pass them into your function as strings. There's one exception, and that's for flags.
- We'll inspect your tasks as much as we can -- the
inspectmodule is extremely powerful, and we'll glean as much as possible about the arg spec, documentation, name, etc. of your function as possible. You shouldn't be burdened by telling us what we can find out programatically - Tasks should be accessible -- whether it's the command-line, through the browser, or through a chat client, the tasks you define should be easily accessed
Like most python projects, build with the included setup.py:
python setup.py install
Currently there are two dependencies:
# You'll need argparse
pip install argparse
# If you want to run the web app, you'll need bottle, too
pip install bottle
Shovel looks for a file in the current working directory called shovel.py
and executes it to find tasks you've defined (more on that in a second). If
you'd like to modularize your tasks, create a shovel directory, and put
as many python files as you'd like in that directory, and make as many files
as you'd like in there. Do this recursively if you'd like. For example:
shovel/
foo.py
bar.py
testing/
foo.py
zod.py
util/
hello.py
In this way, you can 'modularize' your tasks. By 'modularize,' these are not
full python modules (and we don't currently examing __init__.py in each
directory), but it makes it convenient for organization. Each task is prepended
with the file and directory names. For example, if your shovel/testing/foo.py
defined a task bar, then that task would have the name testing.foo.bar.
If you define tasks in shovel.py instead of using a shovel directory, those
tasks will be in the global namespace. If shovel.py defines a task bar, then
that task's name would simply be bar.
In these python files, use shovel by importing shovel's task decorator. Then, apply as needed:
from shovel import task
@task
def hello(name):
'''Prints hello and the provided name'''
print 'Hello, %s' % name
def not_a_task():
'''Print I'm not considered a task in shovel'''
pass
You can now also keep a ~/.shovel.py file or ~/.shovel directory and to
make tasks globally available.
Invoke shovel with the shovel command. If you would like to know more about
what functions that shovel knows about, ask shovel help or shovel tasks.
If you'd like more information on a specific task or module, you can ask for more information with shovel help. Shovel can figure out lots of things about your tasks. Their names, file, line number, arguments, default arguments, if they take a variable number of parameters, and so forth. When you ask for help on a specific task, everything we know about that task will be presented to you.
# List the tasks in the testing directory
shovel help testing
# Get more help on the testing.test task
shovel help testing.test
Execute tasks with shovel and then the task name
shovel foo.hello
Arguments are passed in a strings, and we really try to give you the same
semantics as when you'd normally invoke a function in python. For example,
arguments are considered positional arguments by default, but you can provide
a keyword name for specificity. For example, to execute foo.bar in a way
equivalent to foo('1', '2', '3', hello='7'), you would invoke it:
shovel foo.bar 1 2 3 --hello 7
Keyword names are merely stripped of the leading dashes when parsed. Also
be warned that shovel options (like --verbose and --dry-run) will not
be available to your function. Speaking of which, if you would like shovel
to be extra talkative (for debugging, perhaps), use the --verbose switch:
shovel --verbose foo.bar 1 2 3 --hello 7
Shovel has a dry-run option that will accept all the parameters you would normally pass into a task, but merely tells you how it would invoke a task. This can be helpful if you want to inspect the arguments that your task would get, to make sure that it's correctly invoked:
shovel --dry-run foo.bar 1 2 3 --hello 7
The one exception to arguments not being interpreted as strings is that
orphan keyword arguments are interpreted as flags meaning 'True.' For example,
if we executed the following, then a and b would be passed as True:
shovel foo.bar --a --b
The reason for this is that flags are common for tasks, and it's a relatively unambiguous syntax. To a human, the meaning is clear, and now it is to shovel.
The shovel utility used to ship with a server for making shovel tasks availble
through the browser, as well as campfire bot. These have now been moved into
their own repos for clarity and modularity:
shovel-server and
shovel-campfire. It's unclear how
much updating will need to be done to those projects, but they can now be
developed independently.
Because typing is no fun, the completions/ directory has information on how
to set up auto-completion with different shells (currently only zsh). Thanks
to philadams for starting this set of helpers!
We had a project that had a fair number of semi-regularly used operational
tasks, and we got sick of copy-and-paste, and we also didn't want to have
a standalone script complete with argparse for each and every one. We didn't
like the alternatives out there, and so, shovel. The original version
constituted a weekend of work, and we've been eating our dog food ever since.
Recently, we realized that a lot of these operational details were intuitive enough that we thought some of our support staff would want to make use of them. Rather than make them keep a copy of the code checked out locally, and use the command line, we figured it would be easiest to make HTTP endpoints for them. That way, we could just add buttons to existing interfaces, and life would be good.
We soon realized that while a nice interface, it's a pain to maintain endpoints
and command line tasks. So, why not make an interface that just runs those
same tasks and does a little bit of presentation to make it a web interface?
So now, without any additional work, you can start up the shovel-server and
have access to all of the tasks you've been using from the command line. In
this way, as developers we can keep one machine up to date and ready to run
code, and still provide access to staff outside of the project.
Pull requests and bug reports are welcome. For bugs, please check that the
issue exists on the master branch before submitting a bug. Also, please
include an example along with the current behavior and the expected behavior.
Bonus points for adding a failing test.
For pull requests, you'll need to add or change tests in support of your proposed change. To run the tests:
nose2 -vThis installs all the packages required to run tests, runs the tests and provides coverage information.