Table of Contents generated with DocToc
nfy
nfy
is an experimental configuration management that aims to unify the concept of the Dockerfile
,
the install script, and the configuration manager.
You can think of it as a bridge between bare metal configuration and containers.
There are no compatibility or reliability guarantees until we're v1.0.0.
Install
go get -u cdr.dev/nfy/cmd/nfy
Introduction
The Problem
- Dockerfiles don't scale.
- As Dockerfiles become large, developers begin moving installation to scripts and installers, undermining layer caching.
- Smaller files are more readable.
- Small scripts can be tested independently.
- As Dockerfiles become large, developers begin moving installation to scripts and installers, undermining layer caching.
- Dockerfiles are constrained by their linear import graph.
- Software tool images (e.g rust, nginx) force an operating system on the user.
- Multiple images can't be used at the same time.
- If you want an image with
go
andrust
, you use one as yourFROM
, and writeRUN
statements to simulate what the image already does.
- If you want an image with
- Dockerfiles can't be applied to the local system.
- Development workstations are long-lived and bare metal, where production servers are ephemeral and containerized. Software management diverges greatly between the two.
The Goal
- Containerization
- Build container images from your install scripts.
- Update software in environments without recreating the environment.
- Run
nfy install
to apply the configuration to your local machine. - Run
nfy build
to create a container with the install scripts.
- Run
- Speed
- Automatically parallelize installation.
- Stop duplication computation by coupling each install target with an explicit check.
- Portability
- Install the same target on different systems based on what requirements are available.
- Reusability
- Share installers and configuration via GitHub imports.
- Use some of your personal dotfiles on your development machine or workstation.
Basic Example
In order to install wget
and vim, create a file called nfy.yml
with the following contents:
wget:
comment: "wget lets us grab files from HTTP servers."
install: "apt-get -y install wget"
check: "wget -h"
# Delegate vim to Ammar's personal dotfiles.
vim:
deps:
- github.com/ammario/dotfiles:vim
then run sudo nfy install
, which installs every target by default.
If wget
is already installed, the check
step will pass and install
won't run.
Build Container Image
Run sudo nfy build -b ubuntu nfy-ubuntu
to build an Ubuntu container image called nfy-ubuntu
with wget
and my vim
configuration.
The Dockerfile looks like:
FROM ubuntu
# wget: wget lets us grab files from HTTP servers.
RUN apt-get -y install wget
Advanced Example
apt-get:
check: "apt-get -h"
apt-update:
comment: "Ensure the package cache is up to date."
install: "apt-get update -y"
build_only: true
deps:
- apt-get
apt:
deps:
- apt-get
- apt-update
htop:
install: "apt-get install -y htop"
check: "htop -h"
deps:
- apt
wget:
install: "apt-get install -y wget"
check: "wget -h"
deps:
- apt
produces
FROM ubuntu
# Ensure the "apt-get" dependency exists:
RUN apt-get -h
# apt-update: Ensure the package cache is up to date.
RUN apt-get update -y
RUN apt-get install -y wget
RUN apt-get install -y htop
(those comments are generated automatically)
Parallelism
nfy
creates a tree of files and external dependencies rooted in your nfy.yaml
. The tree is a directed acyclic graph
and evaluated with automatic parallelism. Take this example (X -> Y
means X depends on Y):
A -> B
A -> C
B -> D
Is converted into 2 threads that look like:
Compute D
Wait C
Compute B
Compute C
Wait B
Compute A
This parallelism is active during apply and build, meaning nfy
can perform much faster than Docker.
Recipes
Each recipe has a name or a target. For example:
curl:
install: "..."
has the target curl
.
The target can be used to reference the recipe later.
Recipes support the following parameters:
Name | Usage |
---|---|
install | An executable command or script to install. |
check | An executable command or script location to check if installation is necessary. |
fast_install | If "yes", indicates that the install is fast enough and running check is unnecessary. |
deps | A list of targets which must exist before this can install. |
build_only | Specify whether command will only run in container builds. |
comment | Include a comment in the Dockerfile. |
files | A list of files which must be available in the working directory. |
A target must implement one of check
or install
. A target with install
and no check
will print a warning when
it is evaluated, unless fast_install
is set.
A target with a check
but no install can be used to represent hard requirements, such as
# Ensures apt-get exists
apt-get:
check: "apt-get -h"
Code Structure
Import Statements
An import
directive loads in recipes from a file. The configuration is evaluated in order, so the list of imports
must be before the imported target is referenced. Example:
import:
- "apt.yml"
htop:
install: "sudo apt -y install htop"
deps:
- apt
Every import is processed before recipe evaluation begins.
You can also import entire directories via globbing. For example:
import:
- "nfy/*.yml"
Dependencies
Dependencies can exist on a per recipe and per file basis. Dependencies on a file are automatically added to each recipe in the file.
Dependencies in the nfy.yml
file can make hard, global requirements about the system, for example:
apt-get:
check: "apt-get -h"
deps:
- apt-get
Target Evaluation
A target can be provided in one of three formats:
wget
references a local target somewhere in the source treegithub.com/user/repo:wget
references a remote target namedwget
hosted on git.
Target Overloading
What if you want to install htop
in your Macbook or Linux server?
htop:
check: "htop -h"
install: "apt-get install -h htop"
deps:
- apt-get
will obviously fail because apt-get is not installed. Instead, you should overload the target with multiple installers. For example:
htop:
check: "htop -h"
install_apt:
script: "apt-get install -y htop"
deps:
- apt
install_brew:
script: "brew install htop"
deps:
- brew
We cannot simply provide multiple install
directives because it is illegal YAML for keys to conflict.
The suffix is nice for debugging nfy execution, too.
Use Cases
- Targetting multiple operating systems
- Tolerating different dependencies (e.g using "curl" instead of "wget")
Docker
The first target is always used in Docker images. The first is assumed the default.
Locking
Unimplemented
Each external dependency is locked to a particular commit in the nfy.lock
file.
You can update the repository with nfy update github.com/<user>/<repo>@<tag>
.
The locking mechanism offers security and stability to your config. You should check in your lock file.