This suite of "quick and dirty" scripts are intended to mount virtual sandboxes for labs during Linux training courses with KVM/libvirtd (on Centos 7/8, Ubuntu Bionic 18.04 or Debian stable hosts). The main goal is to spawn quickly linux guests for lab purposes. But we can find other subsequents objectives as coding/scripting, automation or virtual networks and system management, helping to understand virtual infrastructures technologies and architectures. Only for educational purposes.
Three groups of scripts :
- Native installation for Centos 7/8 Debian Stable and Ubuntu Bionic 18.04 and post-installation.
- Quickbuilder, guests spwaning with pre-builded images
- Devices management
If you already have images at stock, there exists better ways to automate and manage libvirt guest like those nice scripts :
- in python :
kcli(https://github.com/karmab/kcli) - in bash : https://github.com/vpenso/libvirt-shell-functions
For native installation of Arch Linux : look at https://github.com/Anthony25/spawn-archlinux-libvirt.
Purposes : gold image auto-creation
autoprep.sh: prepare your system as virtualization hostdownload-images.sh: get a builded image by this projectauto-install.sh: build by yourself a fresh Centos, Debian or Ubuntu system with http repos and kickstart filesauto-install-tui.sh: auto-install.sh text user interface demosparsify.sh: optimize space disk on the designated guestclone.sh: clone as linked, sysprep and optimize builded guestshosts-file: display the running guests and their ipv4 address as /etc/hosts filenested-physical.sh: nested virtualization installation on the physical host
You can also build your images with Packer and Ansible automation based ont this other educational project https://github.com/goffinet/packer-kvm
Some images are available for download on https://download.goffinet.org/kvm : centos7 centos8 fedora32 debian10 bionic focal.
You can download them with the download-images.sh script :
./download-images.sh
-------------------------------------------------------
This script download automatically KVM images
from https://download.goffinet.org/kvm.
Usage:
./download-images.sh image_name [--force]
Where the "image_name" parameter can be:
bionic focal centos7 centos8 debian10 fedora32
The option "--force" does not ask for any confirmation.
Examples:
./download-images.sh bionic --force
./download-images.sh focal
-------------------------------------------------------
As it with user interaction :
./download-images.sh centos7
The image /var/lib/libvirt/images/centos7.qcow2 does not exist.
Do you want anyway download this file centos7.qcow2
Are you sure? [y/N] y
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 611M 100 611M 0 0 123M 0 0:00:04 0:00:04 --:--:-- 123M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56 100 56 0 0 2153 0 --:--:-- --:--:-- --:--:-- 2153
centos7.qcow2: OK
You can force the download for automation purpose :
./download-images.sh debian10 --force
The image /var/lib/libvirt/images/debian10.qcow2 already exists.
The local image is exactly the same than the remote
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 686M 100 686M 0 0 120M 0 0:00:05 0:00:05 --:--:-- 116M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56 100 56 0 0 1696 0 --:--:-- --:--:-- --:--:-- 1750
debian10.qcow2: OK
Purposes : deploy quickly guests based on pre-builded with previous scripts.
define-guest-image.sh: deploy pre-builded images (like a quickbuilder, provisionner) as linked clonesdeploy-image-by-profile.sh: deploy pre-builded images by profiles (xs, s, m, l xl) as linked clonesget-and-install-openwrt.sh: get and start openwrt with two interfaces
But this is probably better to build by yourself your appliance with the auto-install.sh script or with packer and qemu. The root account and the password are stored in the "preseed" and "kickstart" templates included in the script.
Purposes : change RAM and vcpus, add block devices and network facilities
To assign public IPs on VMs hosted on Hetzner hosts see Connect public IPv4 addresses, IPv4 and IPv6 subnets on Hetzner servers.
add-memory.sh: add RAMadd-vcpu.sh: set vcpus countadd-bridge.sh: add an isolated or ipv4 nat/ipv6 ula local libvirt bridgeadd-bridge-l2.sh: create a L2 bridge that connects a physical NICadd-bridge-l3.sh: create a L3 bridge to forward (Hetzner) IPv4 public ranges to your guestsadd-nic.sh: a new NIC on live guest to a bridged interfaceattach-nic.sh: attach a live guest present NIC to a bridgedetach-nic.sh: detach a live guest from a bridgeadd-storage.sh: attach an empty bit disk by GB sizestart_all.sh: start all the defined guestsdestroy-and-undefine-all.sh: destroy, undefine all the guests with storage removing
First clone this project in your home directory :
cd
git clone https://github.com/goffinet/virt-scripts
cd virt-scripts
Script : autoprep.sh
Description : Setup KVM/Libvirtd/LibguestFS on RHEL7/Centos 7/Debian Jessie.
Usage :
# ./autoprep.sh
This script will install all the necessary packages to use Libvirtd/KVM
Please reboot your host after this step
Are you sure? [y/N]
Script : auto-install.sh
Description : Centos 7, Debian Stretch or Ubuntu Bionic fully automatic installation by HTTP Repo and response file (preseed or kickstart) via local HTTP server.
Usage :
Please check all the parameters. For customization, you can adapt the kickstart or preseed templates inside the script.
# ./auto-install.sh
Centos 7, Debian Jessie or Ubuntu Xenial fully automatic installation by HTTP Repos and response file via local HTTP.
Usage : ./auto-install.sh [ centos | debian | ubuntu ] nom_de_vm
Please provide one distribution centos, debian, ubuntu and one guest name: exit
Note : Escape character is :
^](CTRL+]) on Unix french keyboards- CTRL + 5 on Windows french keyboards
Script : sparsify.sh
Description : Sparse a disk. Great gain on disk space !
Usage :
./sparsify.sh
This script sparses an attached disk
Please provide a the guest name of a destroyed guest: exit
Usage : ./sparsify.sh <guest name>
Check the disk usage : 2,0G
# du -h /var/lib/libvirt/images/ubuntu-gold-31122016.qcow2
2,0G /var/lib/libvirt/images/ubuntu-gold-31122016.qcow2
Sparsify operation
# ./sparsify.sh ubuntu-gold-31122016
Sparse disk optimization
[ 0,1] Create overlay file in /tmp to protect source disk
[ 0,1] Examine source disk
[ 4,3] Fill free space in /dev/sda1 with zero
100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ --:--
[ 6,9] Fill free space in /dev/u1-vg/root with zero
100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
[ 70,6] Clearing Linux swap on /dev/u1-vg/swap_1
[ 71,9] Copy to destination and make sparse
[ 191,4] Sparsify operation completed with no errors.
virt-sparsify: Before deleting the old disk, carefully check that the
target disk boots and works correctly.
Check the disk usage : 432M
# du -h /var/lib/libvirt/images/ubuntu-gold-31122016.qcow2
432M /var/lib/libvirt/images/ubuntu-gold-31122016.qcow2
Script : clone.sh
Description : Cloning a domain disk with sparsifying and Linux sysprep.
Usage :
./clone.sh
This script clones, sparsifies and syspreps linux guest
Usage : './clone.sh <original guest> <destination guest>'
Please provide a the guest name of a destroyed guest: exit
Assume that you have installed two guests with auto-install.sh :
~/virt-scripts# ./auto-install.sh centos7 centos
~/virt-scripts# ./auto-install.sh ubuntu1804 ubuntu
And you can verify it :
~/virt-scripts# virsh list --all
Id Name State
----------------------------------------------------
- centos7 shut off
- ubuntu1804 shut off
Undefine original guests :
#!/bin/bash
cd ~/virt-scripts
for x in centos7 ubuntu1804
do
virsh undefine $x
done
And you can deploy quickly builded and optimized guests based on those images :
~/virt-scripts# ./define-guest-image.sh c1 centos7
Début d'installation...
Création du domaine... | 0 B 00:00
Création du domaine terminée. Vous pouvez redémarrer votre domaine en lançant :
virsh --connect qemu:///system start c1
~/virt-scripts# ./define-guest-image.sh u1 ubuntu1804
Début d'installation...
Création du domaine... | 0 B 00:00
Création du domaine terminée. Vous pouvez redémarrer votre domaine en lançant :
virsh --connect qemu:///system start u1
Also, I have prebuilded other images for training classes :
~/virt-scripts# ./define-guest-image.sh
Usage : ./define-guest-image.sh <name> <image>
Please download one of those images :
https://download.goffinet.org/kvm/centos7.qcow2
https://download.goffinet.org/kvm/ubuntu1804.qcow2
You can download them with download-images.sh :
./download-images.sh centos7
The image /var/lib/libvirt/images/centos7.qcow2 does not exist.
Do you want anyway download this file centos7.qcow2
Are you sure? [y/N] y
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 611M 100 611M 0 0 123M 0 0:00:04 0:00:04 --:--:-- 123M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56 100 56 0 0 2153 0 --:--:-- --:--:-- --:--:-- 2153
centos7.qcow2: OK
Verify your running guests :
~/virt-scripts# virsh list
Id Name State
----------------------------------------------------
88 c1 running
89 u1 running
Access to the text console :
~/virt-scripts# virsh console c1
Connected to domain c1
Escape character is ^]
CentOS Linux 7 (Core)
Kernel 3.10.0-514.6.2.el7.x86_64 on an x86_64
centos7 login:
To exit from the text console execute CTRL ].
Script : hosts-file.sh
Description : Print a new /etc/resolv.conf with the ip address and the hostname of running guests.
Usage :
./hosts-file.sh
For example :
./hosts-file.sh
192.168.122.47 c1
192.168.122.118 u1
To update your /etc/hosts :
./hosts-file.sh >> /etc/hosts
SSH is enabled by default :
~/virt-scripts# ssh c1
The authenticity of host 'c1 (192.168.122.47)' can't be established.
ECDSA key fingerprint is 04:be:d2:e9:d9:9a:98:02:e3:a8:34:2d:3a:dd:26:a5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'c1,192.168.122.47' (ECDSA) to the list of known hosts.
root@c1's password:
[root@centos7 ~]# exit
déconnexion
Connection to c1 closed.
Script : add-bridge.sh
Description : add an isolated or ipv4 nat/ipv6 ula libvirt bridge
Usage :
./add-bridge.sh
Description : This script create an isolated or a nat/ipv6 bridge
Usage : ./add-bridge.sh <name> <interface> <type, isolated or nat>
Example : './add-bridge.sh net1 virbr100 isolated' or './add-bridge.sh lan101 virbr101 nat'
Associated scripts :
add-nic.sh: a new NIC on live guest to a bridged interfaceattach-nic.sh: attach a live guest present NIC to a bridgedetach-nic.sh: detach a live guest from a bridge
Script : add-storage.sh
Description : attach an empty bit disk by GB size
Usage :
./add-storage.sh
Description : This script attach a disk to a live guest
Usage : ./add-storage.sh <guest name> <block device name> <size in GB>
Example : './add-storage.sh guest1 vdb 4' add a vdb 4GB disk to guest1
To be continued ... with :
add-memory.sh: add RAMadd-vcpu.sh: set vcpus countstart_all.sh: start all the defined guestsdestroy-and-undefine-all.sh: destroy, undefine all the guests with storage removing
See this document : Connect public IPv4 addresses, IPv4 and IPv6 subnets on Hetzner servers
- (Integrate kcli for guests management)
- Install ansible, add ssh hosts keys, create an ansible inventory and test your managed nodes : see https://github.com/goffinet/packer-kvm
- Exploit virtual storage (LVM, ...)
- Exploit free-ipa, pacemaker, ovirt, openstack, gns3 (see kcli plans)
auto-install.sh- Centos Stream
Centos 8Ubuntu 20.04 Focal Fossa
create_repo.sh: create local repo- Revise code and comment, comment, comment ...