WilliButz/preservation

Error on shutdown unmounting /var/log

r-vdp opened this issue · 5 comments

r-vdp commented

I get the following error on shutdown

sep 07 15:11:57 framework systemd[1]: var-log.mount: Mount process exited, code=exited, status=32/n/a
sep 07 15:11:57 framework systemd[1]: Failed unmounting var-log.mount.

Since it's at shutdown, I can't inspect the system state when this happens. Probably we'd need some ordering to make sure that we only unmount this after journalctl stopped?

I configured it like in the examples, with { directory = "/var/log"; inInitrd = true; }

WilliButz commented

Can confirm this.
The cause for the attempted unmount should be Conflicts=umount.target in the mount units, which is started on shutdown (bootup(7)). I haven't gotten around to figuring out how to handle this case with systemd-journald.service correctly

WilliButz commented

Something like this should precede the above log messages:

systemd[1]: Unmounting var-log.mount...
umount[112539]: umount: /var/log: target is busy.
WilliButz commented

var-log.mount is ordered Before=systemd-journal-flush.service:

→ systemctl show -p Before var-log.mount
Before=systemd-update-utmp.service umount.target systemd-journal-flush.service local-fs.target

meaning it should be stopped after the flush service on shutdown (systemd.unit(5)):

When two units with an ordering dependency between them are shut down, the inverse of the start-up order is applied.

So this should be executed before /var/log is unmounted:

# /etc/systemd/system/systemd-journal-flush.service
ExecStop=journalctl --smart-relinquish-var

From journalctl(1) about --smart-relinquish-var:

This operation is used during system shutdown in order to make the journal daemon stop writing data to /var/log/journal/ in case that directory is located on a mount point that needs to be unmounted.

However, I can't find anything in my logs about the flush service being stopped, which doesn't seem right

r-vdp commented

So, systemd-journal-flush.service does not have Conflicts and Before on shutdown.target anymore since systemd/systemd#27750.
I'm not sure why these were removed, I asked the question on matrix and in the PR (since I can't find the PR author on matrix).

For now, this fixes the issue:

systemd.services.systemd-journal-flush = {
  before = [ "shutdown.target" ];
  conflicts = [ "shutdown.target" ];
};
  • 👍1