no TOTP displayed in systemd-based initramfs

Question

no TOTP displayed in systemd-based initramfs

EvilBit opened this issue 3 years ago · 7 comments

Similar to #74, TOTP is not displayed when using systemd initramfs, i.e. following mkinitcpio.conf hooks on Arch:
HOOKS=(… systemd tpm2-totp …)

I tried writing a small systemd unit file and accompanying installation hook, but no luck.
I checked that all the files are present in the initramfs with lsinitcpio and tried to trace unit execution with systemd-analyze plot.
The unit always gets started way too late in the boot process, long after systemd-ask-password-console.service.

Also, tpm2-totp seemed to fail on first invocation during initramfs stage with:
tpm2-totp[…]: failed to allocate dbus proxy object: Could not connect: No such file or directory
Could this be a side effect of having tpm2-abrmd installed? I saw that it uses dbus and usr/lib/libtss2-tcti-tabrmd.so got included in the initramfs.

On a side note - I haven't researched how to stop the service after the initrd.target. Right now it keeps lingering in the booted system until stopped manually.

Thanks for writing awesome tpm2 tools ;)

For reference, the systemd unit (commented out parameters are from exploring possible permutations):
tpm2-totp.service

[Unit]
Description=Display attestation using tpm2-totp
#Requires=dev-tpm0.device
#After=dev-tpm0.device
Before=systemd-ask-password-console.service
DefaultDependencies=no

[Service]
Type=exec
#ExecStart=/usr/bin/show-tpm2-totp
ExecStart=/usr/lib/tpm2-totp/show-tpm2-totp
#StandardOutput=kmsg+console
StandardOutput=tty

#[Install]
#WantedBy=sysinit.target
#WantedBy=initrd.target

… and a small install hook:
sd-tpm2-totp

#!/bin/bash

build() {
    add_systemd_unit "tpm2-totp.service"
}

NOTE: The unit has to be installed in /usr/lib/systemd/system, not /etc/systemd/system, otherwise the add_systemd_unit function fails silently during mkinitcpio and the unit doesn't get included in the initramfs.

Answer 1 · 2021-07-08T14:44:31.000Z

Thank you for your detailed report! Could you test the following approach, please, which works at least on my machine?

/usr/lib/systemd/system/tpm2-totp.service:

[Unit]
Description=Display a TOTP during boot
Requires=systemd-vconsole-setup.service dev-tpm0.device
After=systemd-vconsole-setup.service dev-tpm0.device
Conflicts=multi-user.target
DefaultDependencies=no

[Service]
Type=exec
ExecStart=/usr/lib/tpm2-totp/show-tpm2-totp
StandardOutput=tty

[Install]
WantedBy=sysinit.target

/usr/lib/systemd/system/sysinit.target.wants/tpm2-totp.service needs to be a symlink to that unit file:

sudo ln -s ../tpm2-totp.service /usr/lib/systemd/system/sysinit.target.wants/

/etc/initcpio/install/sd-tpm2-totp:

#!/bin/bash

build() {
    local mod

    if [[ $TPM_MODULES ]]; then
        for mod in $TPM_MODULES; do
            add_module "$mod"
        done
    else
        add_all_modules /tpm/
    fi

    add_systemd_unit tpm2-totp.service
    add_file /usr/lib/udev/rules.d/*tpm-udev.rules
    add_binary tpm2-totp
    add_binary /usr/lib/libtss2-tcti-device.so.0
    add_binary date
}

/etc/mkinitcpio.conf needs to have sd-tpm2-totp anywhere in HOOKS:

[...]
HOOKS=(base systemd [...] sd-tpm2-totp [...])
[...]

Now regenerate the initramfs and reboot:

sudo mkinitcpio -P

I'll add some more explanation as a separate comment.

Answer 2 · 2021-07-08T14:44:42.000Z

The unit always gets started way too late in the boot process, long after systemd-ask-password-console.service.

Your unit file generally looks fine, but seems to lack a target specification when it should be executed, which might explain why it is executed too late: the service needs to be started as part of sysinit.target, which the commented out WantedBy=sysinit.target is supposed to achieve.

However, mkinitcpio doesn't create the necessary symlink to actually enable it automatically, so you have to create it yourself, see the ln -s command from my first comment. This is also what we do in tpm2-totp for the plymouth service during installation.

Also, tpm2-totp seemed to fail on first invocation during initramfs stage with:
tpm2-totp[…]: failed to allocate dbus proxy object: Could not connect: No such file or directory
Could this be a side effect of having tpm2-abrmd installed? I saw that it uses dbus and usr/lib/libtss2-tcti-tabrmd.so got included in the initramfs.

libtss2-tcti-tabrmd.so gets pulled in by the sd-encrypt hook. This is not really useful since I don't think D-Bus is already up at that point, so tpm2-abrmd wouldn't work anyway. On the other hand, it should only generate an annoying, but harmless error message, then continue to try using libtss2-tcti-device.so.0 instead, which should hopefully succeed. I'll look into getting the sd-encrypt hook fixed; as a workaround, it would be possible to set the environment variable TPM2TOTP_TCTI=device (e.g. by using Environment="TPM2TOTP_TCTI=device" in the unit file) to suppress the error message.

On a side note - I haven't researched how to stop the service after the initrd.target. Right now it keeps lingering in the booted system until stopped manually.

This should be handled by the Conflicts=multi-user.target directive in the unit file above: once user space is reached and this target is active, tpm2-totp.service should get stopped in turn.

Answer 3 · 2021-07-08T15:59:32.000Z

Thanks alot, that fixed it :)

The critical missing thing was actually the manual symlink in /usr/lib/systemd/system/sysinit.target.wants/.
I had enabled the unit using systemctl enable tpm2-totp.service, but that - of course - only created a symlink in /etc/systemd/system/sysinit.target.wants, which is not picked up by the systemd hook (not sure if this behavior is sensible, as well as ignoring units in /etc/systemd/system - It kinda creates two disjunct modes of managing the system with systemd :/)

The first TOTP gets displayed 1-2 seconds after the LUKS password prompt, so still suboptimal, but at least it's there at all.
systemd-analyze plot actually shows that the unit get started correctly before systemd-ask-password-console.service, so that seems to be due to the latency of interacting with the tpm.

On a related note, I'm not entirely sure if StandardOutput=tty is the most general option, as I'm not sure if it also displays correctly when booting over e.g. serial. If someone knows of a more canonical way of displaying output during boot, please let me know.

Oh, and just to clarify: my sd-tpm2-totp install hook was used in addition to the original tpm2-totp hook for testing, so the rest of the original initramfs generation logic was still in place.
BTW, you use the $TPM_MODULES variable - so should the tpm_tis module be added to MODULES=(…) or to this separate variable?

I can try preparing a pull requests in the coming days if you so desire.

Answer 4 · 2021-07-08T16:25:14.000Z

Thanks alot, that fixed it :)

Great to hear :)

The critical missing thing was actually the manual symlink in /usr/lib/systemd/system/sysinit.target.wants/.
I had enabled the unit using systemctl enable tpm2-totp.service, but that - of course - only created a symlink in /etc/systemd/system/sysinit.target.wants, which is not picked up by the systemd hook (not sure if this behavior is sensible, as well as ignoring units in /etc/systemd/system - It kinda creates two disjunct modes of managing the system with systemd :/)

Yeah, that's very confusing indeed - I guess this should be fixed in mkinitcpio as well so that units under the /etc hierarchy are recognised as well.

The first TOTP gets displayed 1-2 seconds after the LUKS password prompt, so still suboptimal, but at least it's there at all.
systemd-analyze plot actually shows that the unit get started correctly before systemd-ask-password-console.service, so that seems to be due to the latency of interacting with the tpm.

Same here, I think it's just the kernel module taking some time to load (which might have a somewhat low priority during boot).

On a related note, I'm not entirely sure if StandardOutput=tty is the most general option, as I'm not sure if it also displays correctly when booting over e.g. serial. If someone knows of a more canonical way of displaying output during boot, please let me know.

I think it should be fine, according to the systemd.exec documentation the standard TTY is /dev/console, so it should also work over serial (but I am by no means an expert).

BTW, you use the $TPM_MODULES variable - so should the tpm_tis module be added to MODULES=(…) or to this separate variable?

The $TPM_MODULES variable controls which kernel modules get included into the initramfs, so you could make the file a little bit smaller by specifying only the modules you need and omit the rest.

In contrast, the mkinitcpio MODULES array controls which kernel modules actually get loaded during boot: this is done by systemd-modules-load.service (which must therefore be started before tpm2-totp.service). It reads configuration files from /etc/modules-load.d/*.conf containing one module name per line and loads them. mkinitcpio automatically creates such a configuration file from the contents of the MODULES array and writes it to /etc/modules-load.d/MODULES.conf.

I am not sure whether there is a better way than specifying the required kernel module statically: usually it is better to just include the necessary modules and let udev to do its job to load them on demand. That's what we do for plymouth-tpm2-totp.service as well (the dev-tpm0.device is automagically generated by systemd once /dev/tpm0 appears), but I haven't been able to get that to work here yet.

I can try preparing a pull requests in the coming days if you so desire.

Sure, that would be great 🎉

Answer 5 · 2021-07-08T16:53:54.000Z

I am not sure whether there is a better way than specifying the required kernel module statically: usually it is better to just include the necessary modules and let udev to do its job to load them on demand. That's what we do for plymouth-tpm2-totp.service as well (the dev-tpm0.device is automagically generated by systemd once /dev/tpm0 appears), but I haven't been able to get that to work here yet.

Never mind, I just forgot to include the udev rules file in the sd-tpm2-totp hook, which is needed to create the dev-tpm0.device device unit. I have updated my first comment, further testing would be much appreciated if you have the time :)

This way, the unit file and install hook look pretty similar to the existing plymouth-tpm2-totp.service.in and sd-plymouth-tpm2-totp.in for plymouth.

Answer 6 · 2021-07-08T17:33:58.000Z

Thanks for the quick replies :)

Never mind, I just forgot to include the udev rules file in the sd-tpm2-totp hook, which is needed to create the dev-tpm0.device device unit. I have updated my first comment, further testing would be much appreciated if you have the time :)

This way, the unit file and install hook look pretty similar to the existing plymouth-tpm2-totp.service.in and sd-plymouth-tpm2-totp.in for plymouth.

Yeah, I realized and implemented that in the meantime as well - taking the plymouth fix from #74 serves as a pretty good skeleton.

Preliminary local PKGBUILD based on tpm2-totp-git from AUR works fine. I'll polish up my code and try to push later today.

Anything special to consider before making a pull request?

Answer 7 · 2021-07-08T17:46:51.000Z

Preliminary local PKGBUILD based on tpm2-totp-git from AUR works fine. I'll polish up my code and try to push later today.

Awesome :) I'll start a new release cycle once this has been merged as well.

Anything special to consider before making a pull request?

Keeping the structure similar to #75 seems like a good idea, otherwise nothing special as far as I can think of - we use the Developers Certificate of Origin, so remember to signoff your commits (git commit -s).

Is the D-Bus/tpm2-abrmd error message still present in your local testing? In that case, I'd include the Environment="TPM2TOTP_TCTI=device:/dev/tpm0" in the unit file as suggested above as a workaround to silence the error until the sd-encrypt hook is fixed. (The unit file already depends on dev-tpm0.device anyway, so the explicit specification of the TPM device is fine.)