Vyatta VC ISO Build Procedures This document describes how to build a Vyatta VC ISO image. The most recent version of this document is the top-level README file from the "latest" branch of the "build-iso" git source repository. Build Host Setup The current Vyatta ISO build requires a Debian "squeeze"-based system. As of now, "squeeze" is the Debian "stable" release. Therefore, to install a minimal squeeze system, download a squeeze installer "netinst" or "businesscard" ISO image from the following Debian page: http://www.debian.org/releases/squeeze/debian-installer/ The supported architectures currently include "i386" and "amd64". During the installation, choose "Standard system utilities" when selecting the software to be installed (note that this is the minimal requirements and one can certainly install more packages). The Vyatta ISO build requires a file system with at least 10GB free space. Allocate a file system of appropriate size, when installing, especially if installing a virtual machine based environment. For development the build file system should have at least 20G free. After the installation and reboot, your "apt" sources.list file should be configured to use the Debian mirror that you selected during the installation. In order to build the Vyatta packages and the ISO, install the following packages using "apt-get" or "aptitude". ssh build-essential sudo bzip2 curl autoconf git devscripts \ debhelper autotools-dev automake libtool bison flex lintian \ libglib2.0-dev libapt-pkg-dev libboost-filesystem1.42-dev \ libncurses5-dev libdb-dev libssl-dev cdbs libmozjs-dev \ libreadline5-dev libpam0g-dev libcap-dev libsnmp-dev gawk unzip \ kernel-package libatm1-dev git-buildpackage libnfnetlink-dev \ libnetfilter-conntrack-dev libattr1-dev rsync libxml2-dev \ libedit-dev libpcap0.8-dev libpci-dev lsb-release quilt ruby \ genisoimage liblzo2-dev unifont libpopt-dev libgmp3-dev \ libcurl4-openssl-dev libopensc2-dev libldap2-dev libkrb5-dev \ hardening-wrapper libgcrypt11-dev libpcre3-dev libprelude-dev \ libgnutls-dev libperl-dev python-all-dev python-setuptools \ live-helper syslinux libsort-versions-perl libexpat1-dev \ libfile-sync-perl gcc-multilib libfreetype6-dev libusb-dev \ libdevmapper-dev libmysqlclient-dev autogen libdumbnet-dev Now install the Vyatta package repository GPG key. wget -O vyatta-pubkey.gpg \ http://packages.vyatta.com/vyatta-dev/vyatta-pubkey.gpg sudo apt-key add vyatta-pubkey.gpg Tools update: You will also need to download and install a newer "squashfs-tools" package from the squeeze-backports repository. Vyatta uses extended file attributes, and older (pre 4.2) versions do not handle attributes. Debian instructions on how to setup squeeze-backports. Add backports to your sources.list 1. Add this line deb http://backports.debian.org/debian-backports squeeze-backports main to your sources.list (or add a new file with the ".list" extension to /etc/apt/sources.list.d/) 2. Run apt-get update After that do: sudo apt-get -t squeeze-backports install squashfs-tools Additional libraries: You will also need to download and install libnetfilter-conntrack3_1.0.0 and libnetfilter-conntrack-dev_1.0.0 packages: PKGS="libmnl0_1.0.3-5_amd64.deb libnetfilter-conntrack3_1.0.0-1+vyatta15_amd64.deb libnetfilter-conntrack-dev_1.0.0-1+vyatta15_amd64.deb" for p in $PKGS; do wget http://packages.vyatta.com/vyatta-dev/pacifica/unstable/pool/main/$p done dpkg -i $PKGS Getting "build-iso" Source The Vyatta ISO builder consists of scripts and config files maintained as a git repository named "build-iso". First, clone the build-iso git repository. git clone http://git.vyatta.com/build-iso.git This will create the source tree in the directory "build-iso". (At this point, you might want to switch to reading ./build-iso/README, which is the most recent version of the file which you are reading now:-) Our git repositories have branches, which usually correspond to different releases. To switch to a particular branch (let's call it BRANCH), do the following. cd build-iso git checkout --track -b BRANCH origin/BRANCH Build ISO To build a Vyatta ISO, do the following in the "build-iso" directory (which has been set up as described in the previous section). export PATH=/sbin:/usr/sbin:$PATH autoreconf -i ./configure sudo make iso The end results is the following livecd ISO image. livecd/binary.iso Note that the above build process uses the pre-built binary packages from the Vyatta "unstable" package repository, which contains the latest development packages updated in real time (i.e., they are the exact same packages used in Vyatta internal development build). If you want to make modifications to one or more packages, you will need to get the source of the submodule(s) that you want to change, make the changes, build the submodules, and then build a new ISO to include the new packages. This is discussed in the next section. Build Host Setup for Building Submodules Before building any submodules, you will need to install the following Vyatta-specific packages on the build host. From "Debian lenny" install the following packages: libnet0 libnet0-dev libsablot0 libsablot0-dev The Debian lenny packages are available from any Debian mirrors. For example, the amd64 "libnet0" package can be found at the following URL: http://packages.vyatta.com/debian/pool/main/libn/libnet0/libnet0_1.0.2a-7_amd64.deb From "Debian wheezy" install the following packages (similarly, they are available from Debian mirrors): libmnl0 libmnl-dev From Vyatta "unstable" package repository install the following packages: iptables iptables-dev linux-vyatta-kbuild libsnmp15 libsnmp-dev libsnmp-perl libsnmp-base libcap-dev libcap2 libvyatta-cfg-dev libvyatta-cfg1 libdaq0 libdaq-dev The location of the Vyatta "unstable" package repository depends on the source branch. For example, for the "pacifica" branch, the amd64 "iptables" package in the "unstable" package repository can be downloaded from the following URL (note that the unstable package repository is updated in real time, so latest version may be different from the version below): http://packages.vyatta.com/vyatta-dev/pacifica/unstable/pool/main/iptables_1.4.10+vyatta+5_amd64.deb After downloading the packages listed above, they can be installed using "dpkg". Then put the following in "/etc/apt/preferences" to "pin" the version of the libcap-dev and libcap2 package (which ensures that the Vyatta package will be used instead of the one from Debian). cat >| /etc/apt/preferences.d/libcap-dev <<EOF Package: libcap-dev Pin: version 2.* Pin-Priority: 1001 EOF cat >| /etc/apt/preferences.d/libcap2 <<EOF Package: libcap2 Pin: version 2.* Pin-Priority: 1001 EOF Getting Submodules Source and Building Note that the "build-iso" git repository contains a number of submodules in the "pkgs/" subdirectory. To get the source for a submodule, do the following in the "build-iso" directory (replace "SUBMODULE" with the name of the module, e.g., "vyatta-cfg-quagga"). git submodule init git submodule update pkgs/SUBMODULE The above will clone the source of the submodule into the "pkgs/SUBMODULE" directory. Note that the submodule needs to be on the same branch as "build-iso", so do the following (replace "BRANCH" with the branch name you used for build-iso). cd pkgs/SUBMODULE git checkout --track -b BRANCH origin/BRANCH In most cases, you will only update (i.e., clone) the submodule(s) that you are going to change. If you want to clone all submodules, do the following. git submodule update Remember to create and checkout the same working branch for all submodules. Build submodules When you do a "make", the build system will build the binary packages for all cloned submodules and include them in the ISO (pre-built packages from the Vyatta "unstable" package repository will be used for the submodules that are not cloned). If you want to work with specific submodules, you can use the commands listed below. Using make targets There are a number of top-level make targets for working with submodules. The commands below use submodule "vyatta-cfg" as an example. To build ".deb" packages for the vyatta-cfg submodule (equivalent to "debuild -i -b -uc -us -nc"): make vyatta-cfg To clean the build for the vyatta-cfg submodule (equivalent to "debuild clean"): make clean-vyatta-cfg To do a "release" for the vyatta-cfg submodule (equivalent to "pkg-release -p", i.e., auto-generate everything and no confirmation): make release-vyatta-cfg Using submod-mk script The submod-mk script in tools/ can also be used to work with submodules. To build binary packages for cloned submodule(s), tools/submod-mk { SUBMODULE... } To build binary and source packages for submodule(s), tools/submod-mk --source { SUBMODULE... } To build signed binary and source packages for submodule(s), tools/submod-mk --signed-source { SUBMODULE... } To clean files generated during submodule build, tools/submod-clean SUBMODULE... To clean files generated during all submodule builds, tools/submod-clean To also clean the submodule build stage(s), tools/submod-clean -d [ SUBMODULE... ] To clean the source tree of submodules to skip subsequent builds, tools/submod-clean -g [ SUBMODULE... ] This retains the local bare git repository for quicker updates. Changes After making, testing, and committing your changes to your local repositories, publish them to the shared repository with, git push git.vyatta.com:/git/SUBMODULE.git This will, of course, require an account on git.vyatta.com, which is only available to authorized vyatta developers. Then update and publish the super-module index to include the submodule changes, cd ../.. git update-index pkgs/SUBMODULE git commit -m 'update sub-module SUBMODULE' git push git.vyatta.com:/git/build-iso.git Updates using git Update the super-module. git pull git submodule init To update all modules, you can use: tools/submod-git pull Or for the paranoid tools/submod-git fetch git whatchanged HEAD..origin/master git merge To see which checked-out submodules have upstream modifications, git status # search for changes in pkgs/ Get the "from" and "to" commit ids (SHAs) with, git diff pkgs/SUBMODULE This will produce output something like, diff --git a/pkgs/vyatta-cfg-quagga b/pkgs/vyatta-cfg-quagga index 78124a9..fca1739 160000 --- a/pkgs/vyatta-cfg-quagga +++ b/pkgs/vyatta-cfg-quagga @@ -1 +1 @@ -Subproject commit 78124a9754efe0bda1d9c827c593c5e0635fec3c +Subproject commit fca1739f1985c2ba8d6316b1882365f5e26245cd To review what changed between these SHAs, cd pkgs/SUBMODULE git whatchanged FROM_SHA TO_SHA To merge these submodule changes, git merge cd ../.. git submodule update The "git status" for this submodule should now be clean. Iterate through the other updated submodules, then rebuild the ISO.