Running Ubuntu cloud images with user-data
mustafaakin opened this issue · 10 comments
I am trying to run the Ubuntu Cloud images with hyperkit. My script is as follows:
KERNEL="xenial-server-cloudimg-amd64-vmlinuz-generic"
INITRD="xenial-server-cloudimg-amd64-initrd-generic"
CMDLIN="earlyprintk=serial console=ttyS0 root=/dev/sda1"
IMAGE="xenial-server-cloudimg-amd64-disk1.img"
QCOW2="xenial-server-cloudimg-amd64-disk1.qcow2"
hyperkit -A -m 512M -s 0:0,hostbridge \
-s 31,lpc \
-l com1,stdio \
-s 1:0,ahci-hd,file://$(pwd)/$QCOW2,format=qcow \
-s 5,ahci-cd,$(pwd)/seed.img \
-f kexec,$KERNEL,$INITRD,$CMDLIN
To convert it to a proper qcow2 format.
qemu-img convert -O qcow2 $IMAGE $QCOW2
qemu-img resize $QCOW2 +50G
By default, it tries to read a cloudinit metadata from network and fails. I instead use the cloud-localds on Ubuntu to generate user-data and meta-data files in a CD-ROM (ISO image). However I get the following output and it hangs while trying to read from that ATA device.
How it is done on Ubuntu with KVM can be found here: http://blog.dustinkirkland.com/2016/09/howto-launch-ubuntu-cloud-image-with.html
[ OK ] Created slice system-getty.slice.
[ OK ] Found device /dev/ttyS0.
[ OK ] Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
hyperkit: [INFO] Allocator: 32764 used; 0 junk; 0 erased; 0 available; 0 copies; 0 roots; 0 Copying; 0 Copied; 0 Flushed; 0 Referenced; max_cluster = 32767
hyperkit: [INFO] Allocator: file contains cluster 0 .. 32767 will enlarge file to 0 .. 33279
hyperkit: [INFO] resize: adding available clusters (Node ((x 32768) (y 33279) (l Empty) (r Empty) (h 1) (cardinal 512)))
hyperkit: [ERROR] write sector = 2004992 length = 4096: I/O deadline exceeded
hyperkit: [INFO] (((description "Anonymous client 6305")
(locks (((description "cluster 1") (mode Read) (released false)))))
((description "write sector = 2004992 length = 4096")
(locks
(((description "cluster 6874") (mode Read) (released false))
((description "cluster 14644") (mode Read) (released false))))))
[ 39.012675] ata1.00: exception Emask 0x0 SAct 0x7fffffff SErr 0x0 action 0x6 frozen
[ 39.013944] ata1.00: failed command: WRITE FPDMA QUEUED
[ 39.014795] ata1.00: cmd 61/00:00:c0:78:80/01:00:04:00:00/40 tag 0 ncq 131072 out
[ 39.014795] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[ 39.017010] ata1.00: status: { DRDY }
I tired creating ISO files from Mac directly and it did not help as well:
hdiutil makehybrid -iso -joliet -o image.iso cidata
seed.img is genereated as follows:
$ cat seed
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
ssh_import_id: kirkland
$ cloud-localds seed.img seed
$ file seed.img
seed.img: ISO 9660 CD-ROM filesystem data 'cidata'
$ file image.iso --> i created it
image.iso: ISO 9660 CD-ROM filesystem data 'CIDATA'
But I guess they are both valid ISO files since I can easily mount both of them in Mac and read the contents.
could you post the full log. We don't normally boot from disk as we normally either boot EFI ISOs (which requires the OVMF firmware) or kernel+initrd directly.
I do not boot from disk, it boots from -f kexec,$KERNEL,$INITRD,$CMDLIN. ISO file is only for providing secondary disk, to read the metadata.
Output is here:
https://gist.github.com/mustafaakin/4f6b56f11a3762099aca958160a10a0d
I would like to help anyway I can if you point me to right directions
@djs55 Looks like the ocaml qcow stuff is unhappy with this image:
hyperkit: [INFO] Allocator: 32764 used; 0 junk; 0 erased; 0 available; 0 copies; 0 roots; 0 Copying; 0 Copied; 0 Flushed; 0 Referenced; max_cluster = 32767
hyperkit: [INFO] Allocator: file contains cluster 0 .. 32767 will enlarge file to 0 .. 33279
hyperkit: [INFO] resize: adding available clusters (Node ((x 32768) (y 33279) (l Empty) (r Empty) (h 1) (cardinal 512)))
hyperkit: [ERROR] write sector = 2004992 length = 4096: I/O deadline exceeded
hyperkit: [INFO] (((description "Anonymous client 6305")
(locks (((description "cluster 1") (mode Read) (released false)))))
((description "write sector = 2004992 length = 4096")
(locks
(((description "cluster 6874") (mode Read) (released false))
((description "cluster 14644") (mode Read) (released false))))))
Any ideas?
This is my incantation:
sudo hyperkit -m 512M -l com1,stdio -c 1 -s 31,lpc -s 0:0,hostbridge \
-s 1:0,ahci-hd,file:///PATH_TO/ubuntu-16.04-server-cloudimg-amd64-uefi1.qcow2,format=qcow \
-s 1:1,ahci-cd,PATH_TO/cloud-init-config.iso \
-f kexec,PATH_TO/ubuntu-16.04-server-cloudimg-amd64-vmlinuz-generic,PATH_TO/ubuntu-16.04-server-cloudimg-amd64-initrd-generic,"earlyprintk=serial console=ttyS0 root=/dev/sda1 rw"
where the disk image is the ubuntu img converted to decompressed qcow2 form (using the same qemu-img command you use). Verify the output with:
$ qemu-img info ubuntu-16.04-server-cloudimg-amd64-uefi1.qcow2
image: ubuntu-16.04-server-cloudimg-amd64-uefi1.qcow2
file format: qcow2
virtual size: 2.2G (2361393152 bytes)
disk size: 954M
cluster_size: 65536
Format specific information:
compat: 1.1. <- note this!
lazy refcounts: true
refcount bits: 16
corrupt: false
It boots just fine.
How did you generate the cloud-init iso image? My Ubuntu image is fine, my script already does that conversion as well
$ qemu-img info xenial-server-cloudimg-amd64-disk1.qcow2
image: xenial-server-cloudimg-amd64-disk1.qcow2
file format: qcow2
virtual size: 52G (56048484352 bytes)
disk size: 2.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
The ISO is a raw disk:
$ qemu-img info image.iso
image: image.iso
file format: raw
virtual size: 900K (921600 bytes)
disk size: 900K
I noticed you used the uefi based image, did not thought it would matter since the booting is handled via hyperkit in kexec, but the ubuntu-16.04-server-cloudimg-amd64-uefi1.qcow2 works perfectly. Not sure why it happens though. Sorry for keeping you busy, great work & tool thanks for your efforts!
Using UEFI image shouldn't be necessary, it just happened to be the last one I tried! I've definitely used ubuntu-16.04-server-cloudimg-amd64-disk1.img in the past with success.
I have a faint recollection that trying to boot from the compressed qcow2 disk image would corrupt it. Perhaps that happened?