Remove (or set to false):
services.xserver.displayManager.startx.enable = true;
Add:
services.xserver.desktopManager.xfce.enable = true;
services.spice-vdagentd.enable = true;
After adding a font package, run:
fc-cache -fHome-manager doesn't do this automatically.