INIMA stands for INIt MAchine (it also apparently means heart in Romanian). It is a collection of scripts, notes and references on how to configure operating systems (primary modern windows and debian-based systems).
Setting up a computer or virtual machine always takes a lot of effort and time, it’s easy to forget to configure something, and after a couple of weeks noone knows how the machine has be configured.
On most Linux systems it is easy to automate most of the stuff, so I’ve begun to collect various command used for setting my environments up.
In the last years, tools like docker and packer emerged, providing the ability to ease the creation of virtual environments.
I decided therefore to resurrect, polish and update some of my old scripts and notes.
Templates are used by packer, together with the bootstrap directory it should be possible to create automatically a virtual machine.
Collections of functions for setting an environment. They can be used with packer, docker, vagrant, on a virtual environment or real machine.
The scripts should also work well for a new machine, as for an already provisioned one. Most functions expect to be running with admin rights.
Functionalities are splitted in functions, in order to be able able to invoke them from the command line too.
packer build template/virtualbox/debian_base.json packer build template/virtualbox/debian_packages.json vboxmanage import out/virtualbox/debian_packages/debian.ovf
packer build template/virtualbox/windows_base.json packer build template/virtualbox/windows_setup.json vboxmanage import out/virtualbox/windows_setup/windows.ovf
If you want to customize the environment, you should primarily look inside the script directory.
Template and Bootstrap files are as minimal as possible, in order to not depend on a particular configuration or tool.
This has the advantage that the script file have as much logic as possible.
Scripts gives much more liberty where to apply the same logic (install specific packages, enable or disable settings, and so on) on different environments, without having to duplicate them on all templates, and without unnecessary dependencies.
Some settings depend on the environment (like disabling or enabling Drag and drop on a guest system), so you may still need to adapt a template or preseed file to your needs.
Even if it is relatively easy to customize the scripts, some actions, like installing programs, are frequently required.
Supported parameters are:
-
cpus: number of cpus to assign to the virtual machine
-
memory: how much memory, in MB, to assign to the virtual machine
-
headless: boolean parameter that tell packer to show a GUI where the machine is provisioned
-
password: login password/password to set in the
*_basetemplates (not supported on windows templates) -
username: login username/username to set in the
*_basetemplates (not supported on windows templates) -
vm_name: name of virtual machine
-
vmachine: set this value to "linux" or "windows" in order to load the right guest additions, if empty, no additions are loaded (saves a lot of time with windows)
-
packages/choco_packages/cygwin_packages: comma separated list of package to install. Since windows does not have an integrated package manager, use
choco_packagesandcygwin_packages -
keymap: keymap of the virtual machine, formats are host-specific (for example on GNU/Linux the English keyboard is en, on Windows it’s en-US)
-
locale: locale of the virtual machine, formats are host-specific (not currently supported on windows)
-
lc_time: time format of the virtual machine, formats are host-specific (not currently supported on Windows)
-
timezone: timezone of the virtual machine, formats are in the form "area/zone" (host-specific, for example Berlin on Debian is "Europe/Berlin")
-
cleanup: if at the end of the setup we need to clean as much as possible to reduce space requirements. This passage can be time consuming.
The list might not be complete, and some parameter might be available only on some provisioner/gest OS.
Most projects provide just a template that takes an iso, and create a fully fledged Virtual Machine. They might accept tons of parameter for customizing the machine, since one size won’t fit it all.
What those project mostly miss is error handling. I wasted entire days because of windows update. It takes multiple hours to download and apply them. And what happens after you waited four hours and an error happens (for example packer is unable to eject a floppy drive because of an open handle?) Yep, destroy the whole machine and try again!
Saving intermediary result (a naked windows machine, a naked windows machine with updates,…) gives me the possibility not to repeat the whole process, and also test my environments better, since I can inspect different steps one by one.
What if you need quickly a machine for prototyping something and eventually throw it away?
It’s surely irrelevant to wait a couple of hours for cleaning and filling the virtual drive with 0 in order to be able to compress it better and squeeze every little bit out of it.
And if you already have a working environment and just want to install the newest updates without waiting the whole day? Or just add a couple of tools on top of it. Best practice would maybe be to create a completely new machine, but it’s not practicable to always wait hours forgetting a working environment. Why not take an existing machine and add whats missing?
These and other are the reason why there should be at least two packer templates for every machine. Unfortunately this has some overhead since we need to export/import the machine multiple times, but if you are experimenting with those tools, it will save you hours and hours, if not days.
I mostly mentioned Windows, but also with GNU/Linux distributions I had similar problems. You need the LaTeX Suite or want to use a not so minimal desktop environment? You need to download multiple Gigabytes of data, even with a top-speed internet connection it takes some time to download configure and install everything. And yes, sometimes your are not able to download a package (probably the last one), and you need to try a second time.
On the contrary there are not as many script files as in other projects.
I believe that one scripts that does only one thing (install a specific program, change a specific setting, …) for provisioning a VM is not very manageable.
If we want to add or remove an action, we need to edit the template file, add the new script, and eventually edit the bootstrap file.
Of course we should split our functionalities in order that every piece does only one thing, and does it well.
Fortunately both Unix shells and Microsoft powershell have functions, so all functionalities are splitted into functions and called from a "main" script. This way, most of the time, there is no need to speak different languages (shell dialect, packer config files, …) and wonder how we should adapt the whole infrastructure to our needs. Just speak the (power)shell language and look at two files: the one with all functions (they are not that many!) and the main file for calling our functions.
Give a look at https://kb.vmware.com/s/article/1023856 if you are working with vmdk images.
It provides an utility that works both for windows as for linux, defragmenting before compacting can also help:
vdiskmanager -d 'path to vmdk' # defragments image file vdiskmanager -k 'path to vmdk' # compacts image file
Otherwise VirtualBox is able resize vmdk file by converting them to vdi and back (it will change the UUID)
VBoxManage clonemedium disk --format VDI 'path to input vmdk' 'path to output vdi' VBoxManage modifyhd --compact 'path to vdi' VBoxManage clonemedium disk 'path to input vdi' 'path to output vmdk'
While for compressing vdi files with VirtualBox work OOB.
Both windows and the Debian templates have a cleanup setting for removing as much data (logs, documentation, temporary files, …) as possible, and overwrite free space with \0.
Just set the value to true, to enable it.
It is not set by default since it could delete some data useful for a later step, and because overwriting every free byte on the virtual disk is very time and space-consuming.
Also defragmenting and compacting virtual images, is generally a time consuming operation.
Therefore unless an image is going to be used an shared multiple times, probably the saved space is not worth the necessary time, especially for a newly installed operating system.
Also give a look at: http://libguestfs.org/virt-sparsify.1.html Zeroing hard rive is a very long operation.
If you get a error like
Build 'vmware-vmx' errored: Error compacting disk: VMware error: Failed to open the disk '/patho/to/auto detect' : A file was not found (0x4). or Failed to defragment: A file was not found (0x4).
Then it is a known issue (see hashicorp/packer#4885)
A possible solution is to edit the .vmx file and replace the line ide0:0.filename = "auto detect" with ide0:0.autodetect = "true", something like
sed -i 's/^ide0:0.filename[[:space:]]*=[[:space:]]*\"auto detect\"/ide0:0.autodetect = \"true\"/' <vmx file>
should do the job.
I’m sorry, I was not able to test them because of this issue.