Why Bash? Because it's ubiquitious and powerful.
When used correctly, Bash is like sorcery.
But oh! how easy it is to be led astray when guided by the wrong spells. The anwer, of course, lies in the universal school of magic: the School of Copy and Paste.
This repository collects essential Bash incanatations for all your infracoding needs.
The following snippets target Bash version 3, which is very old but is also the version shipped by macOS (thanks Apple!).
Any external tool used in this collection is likely to be preinstalled on most reasonable Unixes (notably macOS and Linux).
Snippets are written in such a way that they can be conveniently pasted into a shell script. They are versioned and look like this:
# start name_of_snippet: v1
# .. code ..
# end name_of_snippet
That way you can easily see where the snippet starts and ends. The version number allows you to check if a newer version of this script is available (with improvements or bugfixes). Hopefully updates won't be necessary too frequently.
Add this at the top of your script:
# start our_temp_dir: v1
our_temp_dir=$(mktemp -d 2>/dev/null || mktemp -d -t 'our_temp_dir')
function cleanup_temp_dir() { rm -rf "$our_temp_dir" ; } && trap "cleanup_temp_dir" EXIT
# end our_temp_dir
Then you can use the directory $our_temp_dir
to store files. The directory will be automatically deleted when the script exits.
This works on Linux and macOS. Credit for the macOS trick goes to Krinkle.
Make a habit of starting all your scripts with these two lines:
#!/usr/bin/env bash
set -euo pipefail
This convention is unofficially called Strict Mode. Some peole will say that flags like -e
and -u
are inherently broken. They're not wrong, but Bash is even more broken without these flags (see the Appendix).
Many scripts expect to be called in the context of a project root (usually the directory containing the .git
folder). I call these "situated scripts". In situtated scripts, it's useful to chdir
into the project root at the top of the file, so you can rely on using relative paths throughout your code.
If your script is located in the $PROJECT_ROOT/bin
subdirectory, use the following magical incantation (instead of Strict Mode):
#!/usr/bin/env bash
set -euo pipefail && cd "$(dirname "${BASH_SOURCE[0]}")/.."
Make sure to adjust the relative path (/..
) if the script is located at a different point in your project's folder hierarchy.
Even if changing the project root is not the right choice for your script, it can still be useful to use paths relative to the directory containing the script:
our_project_root="$(dirname "${BASH_SOURCE[0]}")/.."
Again this assumes that the script is located in a direct subfolder of the project root. If that's not true in your project, change the /..
part accordingly.
Sometimes you need an absolute path to your project root:
our_abs_project_root="$(python -c "import os,sys; print os.path.abspath(sys.argv[1])" "$our_project_root")"
The abspath
operation ensures that the path starts with a slash. We use python here because it's available on both macOS and Linux. You can use os.path.realpath
instead of os.path.abspath
if you need to resolve symbolic links in addition to finding an absolute path to the folder. In my experience, however, making paths absolute is usually what you want, not canonizing them.
You need an identifier? You could use a timestamp (accurate to the second):
id="$(date -u '+%Y%m%d%H%M%S')"
or a random UUID:
id="$(python -c 'import uuid; print uuid.uuid4()')"
or generate atomic IDs through an API call (try it):
id="$(curl --silent --show-error https://api.numeri.xyz/v1/255c994e-f77f-4ff6-b1d9-edab1f763fdb/increase)"
Command line argument parsing is complicated. Here we opt for a simple and pragmatic approach. This snippet supports
- flags like
--help
or-h
- and arguments like
--path PATH
- and any number of positional arguments
But we
- don't support coalescing flags like many Unix commands
ls -Arlt
- or use the
getopt
orgetopts
builtins, which add complexity.
# start our_parse_args: v1
parse_args() {
args=("$@")
positional=()
while [ "${#args[@]}" -gt 0 ]; do
handled=0
if declare -f handle_arg > /dev/null; then
handle_arg
fi
if [[ "$handled" == 1 ]]; then
args=("${args[@]:1}")
else
case "${args[0]}" in
-*)
echo "unknown option: ${args[0]}" >&2
exit 1
;;
*)
positional+=("${args[0]}");
args=("${args[@]:1}")
;;
esac
fi
done
}
# end our_parse_args: v1
To use this snippet, define a function called handle_arg
and set global variables depending on the arguments passed. Positional parameters are stored in the array "${positional[@]}"
. For example:
# set defaults
help=0
foo=bar
handle_arg() {
case "${args[0]}" in
--help)
help=1
handled=1
;;
--foo)
args=("${args[@]:1}") # shift
foo="${args[0]}"
handled=1
;;
esac
}
parse_args "$@"
echo "help=$help, foo=$foo"
echo "positional: ${positional[@]}"
With set -eu
you need to take certain precautions. This works:
[[ -z "${MY_VAR-}" ]] && { echo You need to set MY_VAR >&2; exit 1; }
Yes, the weird ...; }
syntax is necessary because the { }
block is defined in a single line.
You require the binary "wget" to be installed and on the PATH and want to exit with a meaningful error message. Here's how:
command -v wget >/dev/null 2>&1 || { echo "Command wget is not installed." >&2; exit 1; }
/usr/bin/md5sum
? Not installed on MacOS. /sbin/md5
? Not available on Linux. So we use Python, which is preinstalled on Linux and MacOS both.
# Instead of
hash="$(md5sum hello.txt)" # * not portable and prints out the filename
# Use this for md5
hash="$(python -c "import sys,hashlib; m=hashlib.md5(); f=open(sys.argv[1],'rb') if len(sys.argv)>1 else sys.stdin; m.update(f.read()); print(m.hexdigest())" hello.txt)"
# And this for sha256
hash="$(python -c "import sys,hashlib; m=hashlib.sha256(); f=open(sys.argv[1],'rb') if len(sys.argv)>1 else sys.stdin; m.update(f.read()); print(m.hexdigest())" hello.txt)"
The command will work with Python 2 and Python 3. hashlib
also contains other hashing algorithms.
Technically yes:
-e
is hard to predict. It's broken with prefixed variable assignments likeexport x="$(false)"
and (in older versions of Bash) in subshells-u
is broken with empty arrays in Bash 3 (which ships with MacOS)-o pipefail
unfortunately has false positives. It exits in cases where SIGPIPE causes a program to quit but doesn't indicate an error condition:
$ set -o pipefail ; cat /usr/share/dict/words | head -1 ; echo $?
A
141
Nevertheless, I stand by my assessment. In each case bash's behavior is even more broken without the flags. My solution is to avoid problematic constructs, including prefixed variable assignments, empty arrays, and to disable -o
when I expect SIGPIPEs.
Suggest spells by submitting a PR!
Please file an issue if I failed to give appropriate credit for a spell.
This spellbook (c) Paulus Esterhazy pesterhazy@gmail.com