/inaugurate

A generic 'hit-the-ground-running' bootstrap script

Primary LanguageShellOtherNOASSERTION

inaugurate

inaugurate is a generic bootstrap script that optionally supports the immediate execution of the application that is being bootstrapped. You can use it from it's 'official' url with either a hosted, or your own, local application description (as explained below):

curl https://inaugurate.sh | bash -s -- <application> <args>

Or you can download it, optionally customize it (change defaults, add your own application to the script as a default), and host it yourself somewhere.

inaugurate also comes with an 'official app store', although that is, for now at least, mostly to demonstrate it's 'app-store' feature. That app-store, of course, can also be customized and self-hosted.

At the moment, inaugurate is mainly useful if you want to (cross-*NIX-platform) bootstrap Python packages (via pip) that have (non-Python) system dependencies, without the need to have root/sudo permissions (which is accomplished by using conda). Admittedly, a rather limited scope for a self-proclaimed 'generic' thing :-) . I reckon it wouldn't be too hard to extend it to support other types of installs/languages though. Ping me if you have suggestions/use-cases.

For more details, check the relevant sections below:

Examples

So here's an example on how you 'inaugurate' frecklecute, which comes bundled with freckles and lets you run 'declarative' scripts (basically wrapped ansible playbooks). This is a good example of when it's useful to be able to install and run at the same time, since in some cases you might not ever need to run frecklecute again (for example, if you use it to build a Docker container within a Dockerfile):

❯ curl https://inaugurate.sh | bash -s -- frecklecute --help
  'frecklecute' not found in path, inaugurating...
     * checking inaugurate app store for: freckles
     ...
     ... < more inaugurate progress output >
     ...
  Usage: frecklecute [OPTIONS] FRECKLECUTABLE [ARGS]...
     ...
     ... < more usage information >
     ...

Here inaugurate executes the application once it's installed (not doing anything useful, only showing usage information). This behavior (among others) can be turned off, if need be, with the help of an environment variable:

❯ curl https://inaugurate.sh | NO_EXEC=true bash -s -- frecklecute

Of course, we can also use wget if curl is not available on our system:

❯ wget -O - https://inaugurate.sh | SELF_DESTRUCT=true bash -s -- frecklecute --help

In this last example we also use the SELF_DESTRUCT environment variable to instruct inaugurate to delete itself and the application it just installed after that application ran. This might be useful, for example, if you build a container, and want the end-product to be as slim as possible.

By default, inaugurate uses conda to bootstrap the desired application into the users home directory, without needing sudo/root permissions. The user can opt to use sudo though, in which case native systems packages will be installed, and Python packages are installed inside a (individually created) virtualenv. This would look like:

❯ curl https://inaugurate.sh | sudo NO_EXEC=true bash -s -- frecklecute

Features

inaugurate...

  • lets you install (mainly Python, but potentially also other) applications and run them in the same go
  • can (optionally) delete itself and the application it bootstrapped after the command was executed
  • supports 'non-root'-permission installs (via conda)
  • has no dependencies except for either curl or wget (and bzip2 when using conda)
  • creates separate environments for each package it installs (either via Python virtualenv or conda)
  • has it's own 'official' app_store, or lets you use your own, local one
  • can be self-hosted
  • is easily customizable, so can be used for your own application for which you want to provide a bootstrap script
  • supports Debian-, RedHat- based Linux distros, as well as Mac OS X
  • can -- optionally -- also install Mac OS X CommandLineTools for Xcode

Description

inaugurate was written for freckles to enable 'one-line' bootstrap of whole working environments. It turned out to be fairly easy to make it more generic, so it got its own project here. inaugurate (obviously) is not useful for simple cases where you just need to install an application. In 95% of all cases you can do that by just using your system package manager (apt install the-package-you-want).

Some applications require a bit more effort to install (e.g. ansible using pip, although that is getting easier as well). While still being fairly trivial, you need to install some system dependencies, then, if you want to do it properly, create a virtualenv and pip install the package into it. Those are the cases where inaugurate is of some use as it can do those things automatically.

The main reason for writing inaugurate was the aforementioned 'one-line' bootstrap though. Admittedly, I have no idea how often this can be of use for the general public, but I figure its a basic enough pattern that I haven't seen implemented elsewhere (yet -- also I might not have looked well enough), at least not in a generic fashion. So I thought I might as well polish it a bit and put it up for other people to have a look. I imagine there are a few situations where it will make sense. You'll know it when you see it, sorta thing.

Usage

cli

Here's how the commandline interface looks on a high level:

<curl_or_wget_command> https://inaugurate.sh | ENV_KEY_1=<env_value_1> ENV_KEY_2=<env_value> bash -s -- <application> <app_args>
|                    | |                   |   |                                           |           |                      |
 - download command -   ------- url -------     ---------- control behaviour --------------             ---- app execution ---

Or, using sudo:

<curl_or_wget_command> https://inaugurate.sh | sudo ENV_KEY_1=<env_value_1> ENV_KEY_2=<env_value> bash -s -- <application> <app_args>
|                    | |                   |        |                                           |           |                      |
 - download command -   ------- url -------          ---------- control behaviour --------------             ---- app execution ---
download command
either curl (or use curl -s if you don't want to see its progress), or wget -O -
url
always https://inaugurate.sh (you can also use https://freckles.io if you want, though)
control behaviour
see the list below for available options
app execution
this is the same you'd use if you would execute the application if it was already installed and available in your PATH, for example: ansible-playbook --ask-become-pass play.yml

apps descriptions

inaugurate uses text files that describe the requirements that are needed to install an application. This is an example for such a description, for the application ansible:

ENV_NAME=ansible
EXECUTABLES_TO_LINK=ansible ansible-playbook ansible-galaxy ansible-vault ansible-console ansible-doc ansible-pull
EXTRA_EXECUTABLES=
# conda
CONDA_PYTHON_VERSION=2.7
CONDA_DEPENDENCIES=pip cryptography pycrypto git
# deb
DEB_DEPENDENCIES=curl build-essential git python-dev python-virtualenv libssl-dev libffi-dev
# rpm
RPM_DEPENDENCIES=epel-release wget git python-virtualenv openssl-devel gcc libffi-devel python-devel
# pip requirements
PIP_DEPENDENCIES=ansible

By default, inaugurate will check whether the first argument is a path to a locally existing file. If it is, this file will be read. If not, a file named after the provided app name (the first argument to the script) in $HOME/.inaugurate/local-store. If there is, this will be read and the application described therein will be 'inaugurated'. If no such file exists, inaugurate will check whether such a file exists on the official inaugurate app_store.

Here's what the different vars mean:

ENV_NAME
the name of the conda or virtualenv that will be created
EXECUTABLES_TO_LINK
a list of executables that should be linked ot $HOME/.local/bin
EXTRA_EXECUTABLES
an optional list of secondary executables. this is mainly used within freckles. executables in this list are linked into $HOME/.local/share/inaugurate/bin
CONDA_PYTHON_VERSION
if using conda, this is the python version that is used in the new environment
CONDA_DEPENDENCIES
if using conda, those are the packages that will be installed into the new environment
DEB_DEPENDENCIES
if using sudo/root-permissions, and running on a Debian-based platform, those are the packages that should be installed using apt
RPM_DEPENDENCIES
if using sudo/root-permissions, and running on a RedHat-based platform those are the packages that should be installed using yum
PIP_DEPENDENCIES
the python packages to install in the conda or virtualenv environment

downloading inaugurate

As already mentioned, you can either use curl or wget to download inaugurate.sh. Actually, any other tool you have at hand that can download files from the internet, and pipe out their content. I focus on curl and wget since the likelyhood one of them being installed is highest.

curl

As mentioned above, this is how to invoke inaugurate using curl:

curl https://inaugurate.sh | bash -s -- <app_name> <app_args>>

wget

And using wget:

wget -O - https://inaugurate.sh | bash -s -- <app_name> <app_args>

For the following examples I'll always use curl, but of course you can use wget interchangeably.

alternative for interactive command

In case the command you are trying to inaugurate requires interactive input, you can use either of those formats:

bash <(wget -O- https://inaugurate.sh) <app_name> <app_args>

or

bash <(curl https://inaugurate.sh) <app_name> <app_args>

I haven't figured out yet how to do that with sudo though.

sudo/non-sudo

One of the main features of inaugurate is providing the option to install, whatever you want to install, without having to use root or sudo permissions. This only works for applications that are available via conda, or python packages.

The way to tell inaugurate whether to use conda or not is by either calling it via sudo (or as root user) or as a 'normal' user. In the former case inaugurate will install system packages, in the latter it will install conda (if not already available) and contain all other dependencies within a conda environment.

To call inaugurate using sudo, potentially/optionally using a environment variable to control its behaviour, you do something like:

curl https://inaugurate.sh | sudo NO_EXEC=true bash -s -- frecklecute --help

environment variables

Here's a list of environment variables that can be used to change inaugurate's default behaviour, by default all variables are set to false or are empty strings:

NO_ADD_PATH
if set to true, inaugurate won't add $HOME/.local/bin to the path in the $HOME/.profile file
NO_EXEC
if set to true, inaugurate won't execute the inaugurated application after install
FORCE_CONDA
if set to true and run as user 'root', inaugurate will use 'conda' (instead of system packages). This doesn't have any effect if used in combination with 'sudo'
FORCE_SUDO
if set to true and not run as user 'root' or using 'sudo', inaugurate will not run
FORCE_NON_SUDO
if set to true and run using 'sudo', inaugurate will not run
SELF_DESTRUCT
if set to true, inaugurate will delete everything it installed in this run (under $HOME/.local/share/inaugurate)
INSTALL_BASE_DIR
if set, inaugurate will install under the specified directory. if not set, default install dir is $HOME/.local/share/inaugurate
PIP_INDEX_URL
if set, a file $HOME/.pip/pip.conf will be created, and the provided string will be set as as index-url (only if pip.conf does not exist already)
CONDA_CHANNEL
if set, a file $HOME/.condarc will be created, and the provided string will be set as the (sole) conda channel (only if .condarc does not exist yet)
CHINA
if set to true, PIP_INDEX_URL and CONDA_CHANNEL will be set to urls that are faster when used within China as they are not outside the GFW, also, this will try to set debian mirrors to ones within China (if host machine is Debian, and inaugurate is run with sudo permissions) -- this is really only a convenience setting I used when staying in Beijing, but I imagine it might help users in China -- if there ever will be any
INSTALL_COMMAND_LINE_TOOLS
if set to true and run with elevated permissions on Mac OS X, inaugurate will make sure that the Mac OS X CommandLineTools are installed. this was required before inaugurate used the get-pip.py script to install pip on Mac

How does this work? What does it do?

inaugurate is a shell script that, in most cases, will be downloaded via curl or wget (obviously you can just download it once and invoke it directly). It's behaviour can be controlled by environment variables (see examples above).

inaugurate touches two things when it is run. It adds a line to $HOME/.profile, and it creates a folder $HOME/.local/share/inaugurate where it puts all the application data it installs. In addition, if invoked using root permissions, it will also potentially install dependencies via system packages.

.profile

By default, inaugurate adds those lines to your $HOME/.profile (which will be created if it doesn't exist):

# add inaugurate environment
LOCAL_BIN_PATH="$HOME/.local/bin"
if [ -d "$LOCAL_BIN_PATH" ]; then
    PATH="$PATH:$LOCAL_BIN_PATH"
fi

Obviously, this won't be in effect after your first inaugurate run, as the .profile file wasn't read since then. You can 'force' picking up the new PATH by either logging out and logging in again, or sourcing .profile:

source $HOME/.profile

Adding the inaugurate path to .profile can be disabled by specifying the NO_ADD_PATH environment variable when running inaugurate:

curl https://inaugurate.sh | NO_ADD_PATH=true bash -s -- cookiecutter gh:audreyr/cookiecutter-pypackage

You'll have to figure out a way to manually add your inaugurated applications to your $PATH, or you always specify the full path (e.g. $HOME/.local/bin/cookiecutter).

package install locations

Everything is installed in the users home directory, under $HOME/.local/share/inaugurate. Each application you 'inaugurate' gets its own environemnt (a python virtualenv in case of a sudo install, or a conda environment otherwise). The executables that are specified in the inaugurate app description (for example: https://github.com/inaugurate/store/blob/master/ansible) will be linked into the folder $HOME/.local/bin.

Because everything is contained under $HOME/.local/share/inaugurate, deleting this folder will delete all traces of inaugurate and 'inaugurated' apps (except for the added PATH in .profile) and free up all space (except for potentially installed system dependency packages).

As mentioned, if invoked using sudo (or as user root), inaugurate will try to install dependencies using system packages (and python packages using virtualenv), otherwise conda is used to perform an entirely non-root install. This is the reason why both cases differ slightly in the folders that are created and used:

'sudo'/'root'-permissoins

.local/
 ├── bin
 │    ├── <linked_executable_1>
 │    ├── <linked_executable_2>
 │    ├── <linked_executable_3>
 │    ├── <linked_executable_4>
 │   ...
 │   ...
 │
 ├── share
     └── inaugurate
          ├── bin
          ├── logs
          ├── tmp
          └── virtualenvs
              ├── <one app>
              │      └──bin
              │          ├── <link_target_1>
              │          ├── <link_target_2>
              │         ...
              │         ...
              ├── <other app>
              │      └──bin
              │          ├── <link_target_3>
              │          ├── <link_target_4>
              │         ...
              │         ...

In this case, new application environments are created under .local/share/inaugurate/virtualenvs. So, for example, if you want to activate one of those virtualenvs (something you usually don't need to do as the executables you probably want are all linked into .local/bin which is in your PATH by now), you can do:

source $HOME/.local/share/inaugurate/virtualenvs/<app_name>/bin/activate

deactivate it issuing:

deactivate

'non-root'-permissions

.local/
 ├── bin
 │    ├── <linked_executable_1>
 │    ├── <linked_executable_2>
 │    ├── <linked_executable_3>
 │    ├── <linked_executable_4>
 │   ...
 │   ...
 │
 ├── share
     └── inaugurate
         ├── bin
         ├── conda
         │   ├── bin
         │   ├── conda-meta
         │   ├── envs
         │   │   ├── <one app>
         │   │   │      └──bin
         │   │   │          ├── <link_target_1>
         │   │   │          ├── <link_target_2>
         │   │   │         ...
         │   │   │         ...
         │   │   ├── <other app>
         │   │          └──bin
         │   │              ├── <link_target_3>
         │   │              ├── <link_target_4>
         │   │             ...
         │   │             ...
         │   ├── etc
         │   ├── include
         │   ├── lib
         │   ├── pkgs
         │   ├── share
         │   └── ssl
         └── logs

Conda app environments can be found under .local/share/inaugurate/conda/envs. In this case, if you'd wanted to activate a specific conda environment (again, usually you don't need to do this), you can do:

source $HOME/.local/share/inaugurate/conda/bin/activate <env_name>

and to deactivate:

source deactivate

Is this secure?

What? Downloading and executing a random script from the internet? Duh.

That being said, you can download the inaugurate script and host it yourself on github (or somewhere else). If you then only use app descriptions locally and host your own 'app-store' (or, as those app descriptions are fairly easy to parse and understand, you read the ones the are hosted on the 'official' inaugurate app_store) you have the same sort of control you'd have if you'd do all the things inaugurate does manually.

I'd argue it's slightly better to have one generic, widely-used (not that inaugurate is widely-used at the moment, mind you) and looked upon script, that uses easy to parse configurations for the stuff it installs, than every app out there writing their own bootstrap shell script. inaugurate (possibly in combination with frecklecute to support more advanced setup tasks) could be such a thing, but I'd be happy if someone else writes a better alternative. It's more practical to not have to read a whole bash script every time you want to bootstrap a non-trivial-to-install application, is all I'm saying.

And even if you don't agree with any of this, you could still use a self-hosted inaugurate script for your local or in-house bootstrapping needs. If you have such needs :-)

Since I'm not particularly interested to have the old 'curly bootstrap scripts are evil'-discussion, here are a few links to arguments already made, and fights already fought:

I'm certain inaugurate, as it currently is, could be improved upon, esp. in terms of security and trustworthiness. For example add some sort of easy-to-use gpg-signing feature. As this is only one of a few minor side-projects for me, I can't really spend a lot of time on it at the moment. If anybody feels like contributing I'd be more than happy though!

Create your own, custom inaugurate script

It's as easy as I could possibly make it to adapt the inaugurate shell script for your own application. In order to do this, you need to modify the beginning of the inaugurate script and include the appropriate variable declarations.

Set your own application details

If you want to adapt inaugurate for your own application, you can do that by adding the following variables to inaugurate (read the comments in the file to find the best place for them):

DEFAULT_PROFILE="freckles"
# conda
DEFAULT_PROFILE_CONDA_PYTHON_VERSION="2.7"
DEFAULT_PROFILE_CONDA_DEPENDENCIES="pip cryptography pycrypto git virtualenv"
DEFAULT_PROFILE_EXECUTABLES_TO_LINK="freckles frecklecute freckelize freckfreckfreck frankentree inaugurate frocker"
DEFAULT_PROFILE_EXTRA_EXECUTABLES="nsbl nsbl-tasks nsbl-playbook ansible ansible-playbook ansible-galaxy git"
# deb
DEFAULT_PROFILE_DEB_DEPENDENCIES="curl build-essential git python-dev python-pip python-virtualenv virtualenv libssl-dev libffi-dev"
# rpm
DEFAULT_PROFILE_RPM_DEPENDENCIES="wget git python-pip python-virtualenv openssl-devel gcc libffi-devel python-devel"
# pip requirements
DEFAULT_PROFILE_PIP_DEPENDENCIES="freckles"
DEFAULT_PROFILE_ENV_NAME="freckles"

The most important thing to do is to have a DEFAULT_PROFILE variable set to the name of your package or executable. This indicates to inaugurate that a custom application profile is set. If the executable name that is used by the user in the inaugurate command-line can be found in the DEFAULT_PROFILE_EXECUTABLES_TO_LINK variable value (which, usually, would probably only contain one executable), it'll use the custom profile. If not, it'll try the local and remote app-stores as described above. The meaning of the other vars is the same as is described in apps descriptions (with a prepended DEFAULT_PROFILE).

Hardcode flags/config options

If you want to prevent the user to change or set one of the available environment variables, you can override those like so, in the top part of the inaugurate file:

NO_EXEC=true

Simple, nothing more to it.

Change default behaviour

If you want to change the inaugurate defaults for one or some of the available environment variables, add code like this to the inaugurate file:

if [ -z "$NO_EXEC" ]; then
   NO_EXEC=true
fi

Use luci to create a option-url tree

This is not ready yet, will update details once it is.

Supported platforms

Those are the platforms I have tested so far, others might very well work too. I did my development mainly on Debian-based systems, so other Linux distributions might not (yet) be up to scratch:

  • Linux
    • Debian
      • Stretch
      • Jessie
    • Ubuntu
      • 17.04
      • 16.10
      • 16.04
    • CentOS
      • 7
  • Mac OS X
    • El Capitan
    • Sierra
  • Windows
    • Windows 10 (Ubuntu subsystem) -- not tested/working yet

License

GNU General Public License v3