Let users provide the complete workload executor invocation instead of a standalone executable
prashantmital opened this issue · 2 comments
It seems like many drivers will have to 'wrap' their workload executor scripts in a bash script (or similar) to adhere to the API currently required by astrolabe for workload executors. This design makes it impossible for the 'wrapper script' to set the correct exit status upon being terminated via a SIGINT or CTRL_BREAK_EVENT signal. Consider the following example (*nix):
wrapper.sh (analogous to the workload executor 'wrapper' shell script):
#!/bin/sh
trap "exit 0" INT # We set a handler for SIGINT because if we don't, this shell script will set a non-zero exit status when it receives `SIGINT` irrespective of the exit status of workload.py
python workload.py
workload.py (analogous to a driver's 'native' workload executor script):
import signal
def handler(signum, frame):
exit(2)
signal.signal(signal.SIGINT, handler) # The script will exit with code 2 when it encounters SIGINT
while True:
pass # Usually, we'd do some driver operations here
Now we run the wrapper:
$ ./wrapper.sh
^C% # This is us manually pressing CTRL-C/Command-C on OSX to send SIGINT
$ echo $?
0 # This is the exit status that will be seen by astrolabe
As the example shows, the exit code set by the native workload executor script is lost due to the wrapper script. AFAICT there is no workaround for this, the only alternative being that we stop relying on exit codes entirely.
This change would also make it easier for drivers to integrate astrolabe since this wrapper script might vary between different platforms or runtimes for some drivers. For example, for PyMongo, we should be able to specify the workload executor as follows:
$ astrolabe spec-tests run-one tests/retryReads-resizeCluster.yaml --workload-executor "path/to/pymongovenv/bin/python .evergreen/python/pymongo/workload-executor.py"
which is equivalent to:
$ WORKLOAD_EXECUTOR="path/to/pymongovenv/bin/python .evergreen/python/pymongo/workload-executor.py" astrolabe spec-tests run-one tests/retryReads-resizeCluster.yaml
This would make it much easier to account for a different invocation pattern. In the case of PyMongo, for example, on windows, we could simply do:
$ WORKLOAD_EXECUTOR="path/to/pymongovenv/Scripts/python.exe .evergreen/python/pymongo/workload-executor.py" astrolabe spec-tests run-one tests/retryReads-resizeCluster.yaml
@prashantmital please revert this change, this is not the design we agreed upon