How to diskless boot on a real machine？
593749519 opened this issue · 27 comments
With windrbd-boot.pdf I can surely deploy a diskless client boot on virtual-box, is there any guidance for diskless boot on a real machine, what should be considered to succeed?
Hi 593749519,
Thank you for trying this. It is actually long ago since I touched this piece of code.
What is currently not working on installing the Windows OS on top of a WinDRBD disk is that Windows thinks that a WinDRBD disk is not bootable (which is wrong). I think I found a solution to that and need to do some tests. When I'm done I will let you know via github (this issue).
Thanks again for using WinDRBD,
Best regards,
- Johannes
Hi 593749519,
We are making progress. It looks like UEFI is a requirement, are you able
to boot via UEFI? You can take a look at the comments in
#22 for some info. A user
reported that installing on top of a WinDRBD device partly works.
Hi Johannes, this is Yannis the person who commented on #22.
After testing all scenarios, my conclusion is the following...
Using Windows installer to deploy Windows on a WinDRBD disk (both in BIOS and UEFI boot mode) does not make much sense. Main reason is that, even if you manage to get past of the first stage of the installation (disk partitioning/installing Windows), then second stage of the installation would fail for 2 reasons:
a) In BIOS boot mode, ipxe-windrbd.pxe would load Windows kernel successfully, but eventually it would fail proceeding further due to the fact that there is no static ip address defined on Windows side, and due to the necessary boot services (drivers) not being set to auto start on boot. WinDRBD expects these to be set in order to establish a scucessfull diskless connection with the linux server. Hence, Windows installation cannot proceed any further.
b) In UEFI boot mode, ipxe-windrbd.pxe would fail to load Windows kernel as it's currently not designed to load an UEFI VFAT partition, Windows kernel etc.
What currently works (both on a VM and a physical system) is to prepare the Windows image first either on a VM or a physical system as per instructions described in windrbd-boot.pdf tech guide. Then load the image on the linux server as per instructions on same guide, finally boot the system in BIOS boot mode. This method would not work in UEFI mode.
I believe that having a way to boot Windows via diskless WinDRBD in UEFI mode would make sense, considering that most physical clients (laptops) are currently set to boot in UEFI/Secure Boot mode. BIOS boot mode is slowly becoming obsolete on these systems.
Hi Yannis, of course we know each other :)
I am just now experimenting with stage 2 of the Windows installer booting
from EFI ... one of course needs an iPXE image that supports both EFI
and WinDRBD, then the other stages of the installer (and booting the
final system) should also work.
Regarding static IP address, one has to set up a fixed IP address for
each client in the /etc/dhcp/dhcpd.conf (as also explained in the tech
guide) Windows will use this address when booting so no need to
have a static IP address (this already works for a long time).
I will check the UEFI support of iPXE for the Windows kernel now.
I can imagine that it 'just works' because iPXE is quite feature rich...
I agree that efforts to make Windows boot via PC BIOS and WinDRBD
does not make much sense any more (except for example when
booting ReactOS ...) so I will concentrate on UEFI.
Best regards,
- Johannes
Hi I managed to boot stage2 until the WinDRBD driver (which appears
to be installed correctly) looks for the iBFT/ACPI table in the first 640K
of memory. This is where a PC BIOS puts the tables. It might well be that
UEFI puts the table somewhere else so I need to find a way to read the
boot config from the WinDRBD driver, but I think this can be done.
Yannis, you need a EFI enabled ipxe image: I am attaching it.
The iPXE you have works only with PC BIOS.
You can build your own with:
make bin-x86_64-efi/ipxe.efi
in my branch of iPXE:
https://github.com/johannesthoma/ipxe-windrbd/tree/windrbd
Happy hacking :)
Congrats! There is definitely some progress now. Still it fails booting with "inaccessible_boot_device" BSoD (see below), but I guess that's because you need to find a way to parse the boot configuration to the WinDRBD driver ?
Exactly. WinDRBD does not find the DRBD URL for the resource and does not create a boot disk. PC BIOS stores it in the first 640K so currently WinDRBD just scans the first 640K but EFI does this somehow different. I am working on it :)
Thanks again for your contribution :)
Hi Yannis, I am one step further it looks like the Virtual SCSI Bus Object is not
created in stage2 of the installer...will try to fix this (I hardcoded my URL for now ...).
Thanks Johannes. Quick question, what would be the way to "hardcode" the WinDRBD URL? I understand that this is an alternative method to dynamically provide the URL via the DHCP server?
I hardcoded the URL in the Source Code and rebuild the project (just to see if it would work when it has the URL). This is something I would not recommend since setting up a build environment is unfortunately still not easy and straight forward. If you can wait a few days I hope to find a solution until then (currently working on the missing bus object which in turn causes the INACCESSIBLE_BOOT_DEVICE BSOD).
Sure, no rush on this, I was just curious :)
Hi everybody, made some progress WinDRBD Virtual Bus Device is
now created on stage 2 of the installer. There are some Windows
registry files which need to be applied one has to do this under Linux
(the machine WinDRBD is booting from) since Windows does not
allow writes to the about to installed harddisk image. Actually this
is much easier to accomplish: I will provide a script that does it.
Currently the installing VM (or real machine) has no network need
to find out why ...
Best wishes and happy new year!
- Johannes
Hi Johannes,
This is indeed good news, however I believe that we should not give much effort in making windrbd work with the standard Windows installer, since this can be easily workarounded by having pre-created, syspreped windrbd+windows image which one could use as a base image for any number of physical or VM clients. Syspreped (generalized) Windows images are machine hardware independed, so they can be booted up by any system, no matter their hardware specs. This would also reduce Windows deployment time, as there is no need to repeat Windows installation for each individual deployment.One example would be using for example zfs+clones as Windows image provider. Any progress with the diskless EFI boot ?
Best wishes for the new year!
- Yannis
Hi Yannis,
It looks like the NIC driver doesn't get installed and Windows waits forever for the root device (no BSOD it just loops). I then experimented with Windows preinstall environment (WinPE) and created an ISO image and installed Windows manually (with dism /applyImage and the like) Unfortunately same result. It loos like the driver is in the driver store (E1G something) but does not get loaded. You mentioned syspreped Windows Images. Could you point me into a direction how to set up a Windows image that contains and loads the necessary drivers? (PnP doesn't work with the NIC driver since it requires access to the root device). How is this done with iSCSI targets?
Thanks a lot and best wishes,
- Johannes
Hi Johannes,
Thanks for your continued efforts with this.
I'm not an expert in the area, but I managed to find the following articles which may shed some light, if you haven't seen them already...
http://mistyprojects.co.uk/documents/iscsi-targets/files/intro.htm
All the best,
- Yannis
Hi Johannes,
Managed to find some time to do some further testing on this. I configured an iscsi target (tgt) on same linux VM where DRBD serves the Windows image. I was able to SANBOOT the image via iscsi without any issues, no modification were needed on the image either. I even managed to install WIndows from scratch directly via iscsi (by following the WinPE method in the link I provided on my last post). Still WinDRBD fails with "inaccessible_boot_device" BSOD. My understanding is that something needs to be changed in the "drbd.cgi" script for this to work, but again I'm not an expert :-)
Regards,
- Yannis
Hi Yannis,
Thank you for testing this. To solve the INACCESSIBLE_BOOT_DEVICE BSOD
you have to configure a WinDRBD Virtual Bus Device in the Windows Registry
before booting stage 2. You can do so by mounting the NTFS created by
stage 1 (do a drbdadm primary then you need to kpartx -a and then you
can mount the partition). Then you can use the reged tool to import the
reg files. A script similar to the following should do the trick:
RES=${RES:-windrbd-install-windows7-2}
MINOR=${MINOR:-50}
drbdadm primary $RES
kpartx -a /dev/drbd$MINOR
sleep 2
mount /dev/mapper/drbd${MINOR}p4 mnt/
reged -I mnt/Windows/System32/config/SYSTEM HKEY_LOCAL_MACHINE\\SYSTEM /home/johannes/WinDRBD1.reg -C
reged -I mnt/Windows/System32/config/SYSTEM HKEY_LOCAL_MACHINE\\SYSTEM /home/johannes/WinDRBD2.reg -C
reged -I mnt/Windows/System32/config/SYSTEM HKEY_LOCAL_MACHINE\\SYSTEM /home/johannes/WinDRBD3.reg -C
umount mnt
kpartx -d /dev/drbd$MINOR
drbdadm secondary $RES
Alternatively, if you have a WinPE command line you can also
directly import the registry entries in regedit: select HKLM then you can
Load Hive ... then select Z:\Windows\System32\Config\SYSTEM and name
it SYSTEM-installtarget (see inside the registry file). Then you can
import the reg files. Creating a WinPE generic boot disk is documented
at various places for example at:
However one has to install Windows (and WinDRBD) manually when the ISO
boots (there is no setup.exe) so this method is a bit tricky ...
Attached you find the Windows registry files: they should be readable by
both windows regedit and the Linux reged tool.
Thanks again for trying this,
Best regards,
Hi Johannes,
Many thanks for sharing the .reg files.
I was able to import them in the registry as per your instructions, however I'm still receiving the INACCESSIBLE_BOOT_DEVICE BSOD.
Could it be because my WinDRBD is not the one with the hardcoded DRBD URL ? Did you manage to find a solution for that ?
Thanks again,
- Yannis
Hi Yannis,
Yes, I think you are right. Default WinDRBD driver behavior is to only create a root disk
when the parameters are passed via a (patched) iPXE ... I will build a version that looks
up the URL in the registry so you can experiment with it. I think this is the easiest way
to go (the alternative booting into the stage 2 via iPXE unfortunately does not work yet
for EFI builds).
I will implement that tomorrow and send you a windrbd.sys to be replaced in the file system
(/mnt/Windows/System32/drivers/windrbd.sys), ok? All you need to do then is to set a
registry value to some URL.
Thanks for pointing that out. I will write when I have the WinDRBD.sys ready.
- Johannes
Hi Johannes,
Understood and many thanks again.
- Yannis
Hi Yannis,
As promised a version that allows to configure the WinDRBD root device URL with a registry value. The key is:
HKLM\SYSTEM\ControlSet001\Services\WinDRBD (where also syslog_ip is located) inside that key create a REG_SZ value called BootDeviceURL and assign it the WinDRBD resource URL (the long string starting with drbd:).
To use the driver make the resource primary, kpartx -a and mount it (partition 3 or 4) and copy the
attached sys driver file to mnt/Windows/System32/drivers/windrbd.sys
Then using reged -e create the BootDeviceURL registry value. There should be no BSOD when booting the
stage2 machine instead it waits forever for the network. This is something I haven't solved yet.
windrbd-boot-url-for-yannis.sys.zip
Best regards and happy hacking :)
- Johannes
Thanks Johannes, will test and report back.
- Yannis
Hi Johannes,
Good news, I managed to boot within a VM, a pre-created Win10 image via WinDRBD in EFI mode by using the modified version of windrbd.sys you provided to me (had to disable driver signature enforcement first).
I had also already done all required modifications in the Windows registry beforehand (as they are mentioned in the WinDRBD diskless boot pdf guide). Moreover, I had disabled the Windows pagefile as well, as that was causing issues. So far, so good.
However, when I try to boot the same image from a physical machine (Dell Latitude E7270), it's stuck in the boot process waiting for the network as you mentioned. So clearly, related to the network card drivers not being loaded during the early boot stage.
So far, I have not tried to do a clean Win10 installation directly on a WinDRBD device, but will try that at some point too.
Will continue the tests... :-)
Thanks,
- Yannis
Hi Yannis,
I just tried to install a Windows 7 on top of a BIOS VM with WinDRBD, right now there
is a IRQL_NOT_LESS_OR_EQUAL BSOD I need to get a memory dump in order to fix this
will continue working on it later this week.
But thank you for trying this I think when we solve the no network problem, one would be
able to install atop of a WinDRBD device ... Things I would try also:
-
Maybe it makes a difference from where the Windows CD (stage 1 install) is booted,
maybe it matters whether it is booted from local CD or via network? -
I now disabled the firewall completely for stage 2. Did we try that before?
Best regards,
-- Johannes
Hi Johannes,
I've seen IRQL_NOT_LESS_OR_EQUAL BSOD in the past, when I attempted to install Win10 over WinDRBD. The solution for me was to disable Windows PageFile via registry. Not sure if this is the same case though.
Just wanted to report that I had success booting a Dell laptop over WinDRBD (Win10 image). It has been stable for a couple of days now, even when running disk intensive tasks (I'm running a rw fio job in a loop).
Only "issues" I see so far (when comparing it to iscsi boot) are the following...
-
Windows boot time is much faster on iSCSI. There is a 4-5 min delay when booting over WinDRBD, during the early boot stage (this is a know issue I believe).
-
iSCSI is able to tolerate network failures better than WinDRBD. For example, when I unplug the network cable from the laptop for 5-6 sec, iSCSI can recover the session gracefully, whereas WinDRBD doesn't (when checking DRBD status on Linux side, it stays at "Connecting" status).
Regards,
Yannis