Boot from disk after installation not working

[ika-roa]

Hi!
We tried to set up QEMU to imitate a small industrial PC with two hard drives on a Windows computer (Windows 10 without KVM, and Windows 11 with KVM). The installation from our .iso to the first hard drive worked perfectly fine. However, after the installation, when restarting the docker container, we can't boot from the installed system.

We start the docker container with this modified docker-compose.yml:

version: "3"
services:
  qemu:
    container_name: qemu
    image: qemux/qemu-docker
    environment:
      BOOT: "/storage/boot.iso"
      DISK_SIZE: "16G"
      DISK2_SIZE: "16G"
      BOOT_MODE: "uefi"
      RAM_SIZE: "6G"
      CPU_CORES: "8"
      DISK_FMT: "raw"
      ARGUMENTS: "-smbios type=1"
    volumes:
      - ./storage:/storage
      - ./storage2:/storage2
    devices:
       - /dev/kvm
    cap_add:
      - NET_ADMIN
    ports:
      - 8006:8006
    stop_grace_period: 2m
    restart: on-failure

After installation and restart, the boot manager shows all boot options in the correct order:

(1) UEFI QEMU QEMU HARDDISK (with the installed Debian system)
(2) UEFI QEMU QEMU HARDDISK 2 (empty)
(3) UEFI QEMU QEMU CD-ROM (customized Debian live system with installer).

However, even if we select the first option manually, it always falls back to the installer.

If we use the raw .img files for hard drives 1 and 2 to run QEMU manually in WSL (with Debian 12), we can start up the system without trouble. Only in the docker container it does not work.

As a workaround, we can comment out line 23 from /src/disk.sh (DISK_OPTS="$DISK_OPTS -device ide-cd,drive=cdrom0,bootindex=$BOOT_INDEX"), to get rid of the CD-ROM option. If we do this, we can boot into the installed system as intended.

Do you have any suggestions as to why this could happen? Thanks!

[kroese]

That is very strange.. Because the default BOOT_INDEX setting for the cd-rom is 10. And for the harddrives it is 3 and up. So it should always give the disks higher priority then the cd-rom.

Maybe these indexes are being ignored in UEFI mode, because I tested them with the default SeaBIOS (legacy).

So two questions:

• If you disable UEFI mode, does it solve the problem?
• Can you add the environment variable: DEBUG: "Y" and copy/paste the QEMU commandline parameters here? Maybe by looking at them we can see what is missing or wrong.

[ika-roa]

Thanks for the fast answer! The boot order seems to be in fact correct after the installation, the hard drive has priority over the installer. However, even if I select the hard drive manually in the boot manager, the system falls back to the installer.

Unfortunately, disabling the UEFI option does not solve our issue, as our system needs EFI to boot.

The QEMU command line parameters are as follows:

qemu-system-x86_64 \
  -nodefaults \
  -cpu host,kvm=on,l3-cache=on,migratable=no \
  -smp 8,sockets=1,dies=1,cores=8,threads=1 \
  -m 6G \
  -machine type=q35,graphics=off,vmport=off,dump-guest-core=off,hpet=off,accel=kvm \
  -enable-kvm \
  -global kvm-pit.lost_tick_policy=discard \
  -display vnc=:0,websocket=5700 \
  -vga virtio \
  -monitor telnet:localhost:7100,server,nowait,nodelay \
  -name qemu,process=qemu,debug-threads=on \
  -serial mon:stdio \
  -device qemu-xhci,id=xhci \
  -device usb-tablet \
  -netdev tap,ifname=qemu,script=no,downscript=no,id=hostnet0 \
  -device virtio-net-pci,romfile=,netdev=hostnet0,mac=02:98:4D:F3:0A:5C,id=net0 \
  -object iothread,id=io2 \
  -drive id=cdrom0,media=cdrom,if=none,format=raw,readonly=on,file=/storage/boot.iso \
  -device virtio-scsi-pci,id=scsi0,iothread=io2,addr=0x5 \
  -device scsi-cd,bus=scsi0.0,drive=cdrom0,bootindex=10 \
  -drive file=/storage/data.img,if=none,id=drive-userdata,format=raw,cache=none,aio=native,discard=on,detect-zeroes=on \
  -device virtio-scsi-pci,id=hw-userdata,iothread=io2,bus=pcie.0,addr=0xa \
  -device scsi-hd,bus=hw-userdata.0,channel=0,scsi-id=0,lun=0,drive=drive-userdata,id=userdata,rotation_rate=1,bootindex=3 \
  -drive file=/storage2/data2.img,if=none,id=drive-userdata2,format=raw,cache=none,aio=native,discard=on,detect-zeroes=on \
  -device virtio-scsi-pci,id=hw-userdata2,iothread=io2,bus=pcie.0,addr=0xb \
  -device scsi-hd,bus=hw-userdata2.0,channel=0,scsi-id=0,lun=0,drive=drive-userdata2,id=userdata2,rotation_rate=1,bootindex=4 \
  -device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x4 \
  -object rng-random,id=objrng0,filename=/dev/urandom \
  -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pcie.0,addr=0x1c \
  -smbios type=1

[kroese]

If I look at your parameters, it sets bootindex=3 for the harddisk and sets bootindex=10 for the cd-rom. Just like it is expected. So I dont understand why that does not work.

Maybe if you replace boot.iso by an empty file (so that its a invalid/corrupt iso) it will fallback to the harddrive possibly?

[vejrosta]

Hello,

we tried to replace it with an corrupted iso and it boots now.

[kroese]

Okay, that is good news, but I still cannot explain why you need a corrupt ISO to make it boot.

Maybe this line smbios type=1 that you added is causing the problem.

[vejrosta]

We also tried to delete that with the same results. We also tried to to change the cd rom addr=0x5 to addr=0x10 and higher just to be sure there is no hidden priority conflict on the scsi level as it gave us an impression the UEFI QEMU QEMU HARDDISK is remapped with the cd rom. Also negative.

[kroese]

If you can tell me which QEMU parameters are wrong, I can correct them in the script.

But if none of the QEMU parameters are wrong, than only the QEMU team can fix this bug, because I just launch the VM with parameters that I learned from their documentation. So if their documentation is wrong, there is not much I can do about that.

[kroese]

There is a new version (v5.15) which now supports disk images like qcow2.gz or .img directly in the URL field (not only ISO's anymore).

Furthermore, you can now bind a disk image to /boot.qcow2 or /boot.img. In both cases it doesnt use the CD-ROM at all anymore, so this should solve your issue too!