Time out when use systemd as init
whybeyoung opened this issue · 19 comments
Description of problem
i am using the os-builder to build initrd , when i use the sytemd as the init process, then i call
qemu-system-x86_64 -m 1024 -kernel /boot/vmlinuz-4.15.0-52-generic -nographic -initrd /ext/gopath/src/github.com/kata-containers/osbuilder/rootfs-builder/rootfs/../../initrd-builder/kata-containers-initrd.img -append console=ttyS0 it will fail and stuck in Timed out waiting for device dev-ttyS0.device
Expected result
normally boot.
(replace this text with an explanation of what you thought would happen)
Actual result
stuck in Timed out waiting for device dev-ttyS0.device
(replace this text with details of what actually happened)
this works for me: sudo -E make initrd DISTRO=fedora USE_DOCKER=1 AGENT_INIT=no
Welcome to Fedora 30 (Thirty)!
[ 4.485127] systemd-fstab-generator[37]: Parsing /etc/fstab
[ 4.487587] systemd-gpt-auto-generator[39]: Not a EFI boot, not creating root mount.
[ 4.488119] systemd-gpt-auto-generator[39]: Neither root nor /usr file system are on a (single) block device.
[ 4.493794] systemd-rc-local-generator[41]: /etc/rc.d/rc.local does not exist, skipping.
[ 4.494200] systemd-rc-local-generator[41]: /usr/sbin/halt.local does not exist, skipping.
[ 4.512589] systemd-hibernate-resume-generator[40]: Not running in an initrd, quitting.
[ 4.513904] systemd-bless-boot-generator[34]: Skipping generator, not an EFI boot.
[ 4.518178] systemd-getty-generator[38]: Automatically adding serial getty for /dev/hvc0.
[ 4.519158] systemd-getty-generator[38]: Automatically adding serial getty for /dev/hvc0.
[ OK ] Reached target Swap.
[ OK ] Created slice User and Session Slice.
[ OK ] Listening on Journal Socket (/dev/log).
[ OK ] Created slice system-serial\x2dgetty.slice.
[ OK ] Set up automount Arbitrary…s File System Automount Point.
[ OK ] Listening on Journal Socket.
Mounting Temporary Directory (/tmp)...
[ OK ] Listening on udev Kernel Socket.
[ OK ] Reached target Slices.
[ OK ] Listening on initctl Compatibility Named Pipe.
Starting Rebuild Hardware Database...
[ 4.558720] systemd[46]: tmp.mount: Kernel keyring not supported, ignoring.
Starting Journal Service...
[ OK ] Started Forward Password R…uests to Wall Directory Watch.
[ OK ] Reached target Remote File Systems.
[ 4.563753] systemd[48]: systemd-journald.service: Kernel keyring not supported, ignoring.
[ 4.564297] systemd[47]: systemd-hwdb-update.service: Kernel keyring not supported, ignoring.
[ 4.569348] systemd[48]: Operating on architecture: x86
[ 4.569650] systemd[48]: Operating on architecture: x32
[ 4.576306] systemd[48]: Operating on architecture: x86-64
[ 4.577141] systemd[48]: Operating on architecture: x86
[ 4.577628] systemd[48]: Failed to add utime() rule for architecture x86, skipping: Invalid argument
[ 4.578102] systemd[48]: Operating on architecture: x32
[ OK ] Listening on Process Core Dump Socket.
Starting Create System Users...
[ OK ] Created slice system-getty.slice.
[ 4.586457] systemd[48]: Operating on architecture: x86-64
[ 4.586839] systemd[48]: Operating on architecture: x86
[ 4.587252] systemd[48]: Operating on architecture: x32
[ 4.587653] systemd[48]: Operating on architecture: x86-64
[ 4.587977] systemd[48]: Restricting namespace to: .
[ 4.588242] systemd[48]: Operating on architecture: x86
[ 4.588492] systemd[48]: Blocking cgroup.
[ 4.588667] systemd[48]: Blocking ipc.
[ 4.588851] systemd[48]: Blocking net.
[ 4.589059] systemd[48]: Blocking mnt.
[ 4.589270] systemd[48]: Blocking pid.
[ 4.589570] systemd[48]: Blocking user.
[ 4.589835] systemd[48]: Blocking uts.
[ 4.590124] systemd[48]: Operating on architecture: x32
[ OK ] Started Dispatch Password …ts to Console Directory Watch.
[ OK ] Reached target Paths.
[ 4.591898] systemd[49]: systemd-sysusers.service: Kernel keyring not supported, ignoring.
[ OK ] Reached target Local Encrypted Volumes.
Starting Load/Save Random Seed...
Mounting Huge Pages File System...
[ 4.604209] systemd[50]: systemd-random-seed.service: Kernel keyring not supported, ignoring.
Mounting POSIX Message Queue File System...
[ 4.607516] systemd[48]: Blocking cgroup.
[ 4.607848] systemd[48]: Blocking ipc.
[ 4.608080] systemd[48]: Blocking net.
[ 4.608312] systemd[48]: Blocking mnt.
[ 4.608624] systemd[48]: Blocking pid.
[ 4.608895] systemd[48]: Blocking user.
[ 4.609130] systemd[48]: Blocking uts.
[ 4.609481] systemd[48]: Operating on architecture: x86-64
[ 4.609687] systemd[48]: Blocking cgroup.
[ 4.609882] systemd[48]: Blocking ipc.
[ 4.610126] systemd[48]: Blocking net.
[ 4.611189] systemd[51]: dev-hugepages.mount: Kernel keyring not supported, ignoring.
[ 4.619749] systemd[52]: dev-mqueue.mount: Kernel keyring not supported, ignoring.
Starting Apply Kernel Variables...
[ OK ] Listening on udev Control Socket.
[ 4.630135] systemd[48]: Blocking mnt.
[ 4.630639] systemd[48]: Blocking pid.
[ 4.630934] systemd[48]: Blocking user.
[ 4.631322] systemd[48]: Blocking uts.
[ 4.631764] systemd[48]: Operating on architecture: x86
[ 4.634486] systemd[53]: systemd-sysctl.service: Kernel keyring not supported, ignoring.
[ 4.639639] systemd[54]: systemd-udev-trigger.service: Kernel keyring not supported, ignoring.
Starting udev Coldplug all Devices...
[ OK ] Mounted Temporary Directory (/tmp).
[ OK ] Started Create System Users.
[ OK ] Started Load/Save Random Seed.
[ OK ] Mounted Huge Pages File System.
[ OK ] Mounted POSIX Message Queue File System.
[ OK ] Started Apply Kernel Variables.
Starting Create Static Device Nodes in /dev...
[ 4.679263] systemd[55]: systemd-tmpfiles-setup-dev.service: Kernel keyring not supported, ignoring.
[ 4.692881] systemd[56]: systemd-udev-trigger.service: Kernel keyring not supported, ignoring.
[ OK ] Started Create Static Device Nodes in /dev.
[ OK ] Reached target Local File Systems (Pre).
[ OK ] Reached target Local File Systems.
Starting Restore /run/initramfs on shutdown...
Starting Rebuild Dynamic Linker Cache...
[ 4.711834] systemd[57]: dracut-shutdown.service: Kernel keyring not supported, ignoring.
[ 4.712628] systemd[58]: ldconfig.service: Kernel keyring not supported, ignoring.
[ OK ] Started Restore /run/initramfs on shutdown.
[ OK ] Started Rebuild Dynamic Linker Cache.
[ OK ] Started udev Coldplug all Devices.
[ 4.948257] systemd[48]: Operating on architecture: x32
[ 5.086103] systemd[48]: Operating on architecture: x86-64
[ 5.135822] systemd-journald[48]: Found cgroup2 on /sys/fs/cgroup/unified, unified hierarchy for systemd controller
[ 5.142905] systemd-journald[48]: Journal effective settings seal=no compress=yes compress_threshold_bytes=512B
[ 5.143930] systemd-journald[48]: Fixed min_use=1.0M max_use=99.7M max_size=12.4M min_size=512.0K keep_free=149.6M n_max_files=100
[ 5.146137] systemd-journald[48]: Reserving 22705 entries in hash table.
[ 5.150618] systemd-journald[48]: Vacuuming...
[ 5.151107] systemd-journald[48]: Vacuuming done, freed 0B of archived journals from /run/log/journal/23576a4443704f82b704cde472f21cd6.
[ 5.152071] systemd-journald[48]: Flushing /dev/kmsg...
[ 5.228810] systemd-journald[48]: systemd-journald running as pid 48
[ OK ] Started Journal Service.
Starting Flush Journal to Persistent Storage...
[ 5.242413] systemd-journald[48]: Sent READY=1 notification.
[ 5.243267] systemd-journald[48]: Sent WATCHDOG=1 notification.
[ OK ] Started Flush Journal to Persistent Storage.
Starting Create Volatile Files and Directories...
[ OK ] Started Rebuild Hardware Database.
Starting udev Kernel Device Manager...
[ OK ] Started Create Volatile Files and Directories.
Starting Update UTMP about System Boot/Shutdown...
Starting Rebuild Journal Catalog...
[ OK ] Started Update UTMP about System Boot/Shutdown.
[ OK ] Started Rebuild Journal Catalog.
Starting Update is Completed...
[ OK ] Started Update is Completed.
[ OK ] Started udev Kernel Device Manager.
[ OK ] Reached target System Initialization.
[ OK ] Started Daily Cleanup of Temporary Directories.
[ OK ] Reached target Timers.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System.
Starting Login Service...
Starting Permit User Sessions...
[ OK ] Started Permit User Sessions.
[ OK ] Started Getty on tty1.
[ OK ] Found device /dev/hvc0.
[ OK ] Started Serial Getty on hvc0.
[ OK ] Reached target Login Prompts.
Starting D-Bus System Message Bus...
[ OK ] Started D-Bus System Message Bus.
[ OK ] Started Login Service.
[ OK ] Reached target Multi-User System.
[ OK ] Reached target Graphical Interface.
Starting Update UTMP about System Runlevel Changes...
[ OK ] Started Update UTMP about System Runlevel Changes.
Fedora 30 (Thirty)
Kernel 4.19.86 on an x86_64 (hvc0)
localhost login:
i followed the doc , and not explicitly specify the AGENT_INIT=no , rootfs build comand is script -fec 'sudo -E GOPATH=$GOPATH USE_DOCKER=true EXTRA_PKGS="bash coreutils" ./rootfs.sh ubuntu' , initrd creation command is script -fec 'sudo -E USE_DOCKER=true ./initrd_builder.sh ${ROOTFS_DIR}'
the i use above qemu command , but stuck in timeout of dev-ttyS0.device.
i tried twice , not success, the branch are master and stable-1.10.
i will make a try to use fedora.
@devimc hello , after change the distro to use fedora , it's ok , but still a problem with ubuntu, i think there are sth missing in the initrd with ubuntu. could you have some easy way to find what is missing when the kernel booting?
@berlinsaint sorry I have no idea, what does kernel's log say?
Just like this. @devimc
Actualy i don't know how to get the detail log...
@berlinsaint there are some missing packages in the ubuntu image, try with this:
sudo rm -f .ubuntu_rootfs.done ubuntu_rootfs
sudo -E make initrd EXTRA_PKGS="util-linux p11-kit udev dbus coreutils bash e2fsprogs" DISTRO=ubuntu USE_DOCKER=1 AGENT_INIT=no
@berlinsaint there are some missing packages in the ubuntu image, try with this:
sudo rm -f .ubuntu_rootfs.done ubuntu_rootfs sudo -E make initrd EXTRA_PKGS="util-linux p11-kit udev dbus coreutils bash e2fsprogs" DISTRO=ubuntu USE_DOCKER=1 AGENT_INIT=no
maybe it is indeed. i will trying....
By the way,
Would you mind tell me the way to find this missing packages.... thanks a lot...
Would you mind tell me the way to find this missing packages.... thanks a lot...
... experience/instinct https://gist.github.com/devimc/f8e3373d8d8e6b3b5b56441d04875f2a
pro tip: add a socket console to your qemu command line:
-device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=console.sock
and connect it using socat in other terminal:
socat stdin,raw,echo=0,escape=0x11 unix-connect:console.sock
thanks a lot again @devimc
Would you mind tell me the way to find this missing packages.... thanks a lot...
... experience/instinct https://gist.github.com/devimc/f8e3373d8d8e6b3b5b56441d04875f2a
pro tip: add a socket console to your qemu command line:
-device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=console.sockand connect it using socat in other terminal:
socat stdin,raw,echo=0,escape=0x11 unix-connect:console.sock
emmmm....
i add this and add a virtio-serial-pci, but , it raisedFailed to connect socket console.sock: No such file or directory, who listen this sock? missing sth?
don't add a virtio-serial-pci, the virtconsole should be enough, console.sock is connected to /dev/console in the guest, so you can get the kernel's logs.
I forgot to mention that you may need to add console=hvc0 console=hvc1 to the kernel cmdline
qemu command is: qemu-system-x86_64 -m 1024 -kernel vmlinuz-4.15.0-88-generic -nographic -initrd $ROOTFS_DIR/../../initrd-builder/kata-containers-initrd.img.ubuntu -machine ubuntu,accel=kvm -nographic -no-user-config -nodefaults -append "initcall_debug console=ttyS0 console=hvc0 console=hvc1" \ -device virtconsole,chardev=charconsole0,id=console0 \ -chardev socket,id=charconsole0,path=console.sock,server,nowait
i'am on an virtual machine...have a impact?
and after add the virtio-serial-pci , however ,not output....
ouch! sorry, seems like a serial-pci device is really needed -device virtio-serial-pci,id=serial0
so hard... should i turn on some debug options in kernel?
want to cry......hiahia.. so hard to debug a problemed initrd....
-machine ubuntu,accel=kvm
is ubuntu a new machine type? I use -machine pc or -machine q35
this works for me:
qemu-system-x86_64 -m 2048 -kernel bzImage -daemonize -nographic -vga none -initrd
kata-containers-initrd.img -machine pc,accel=kvm -nographic -no-user-config -nodefaults
-append "initcall_debug console=hvc0 console=hvc1" -device virtio-serial-pci,id=serial0
-device virtconsole,chardev=charconsole0,id=console0 -chardev
socket,id=charconsole0,path=console.sock,server,nowait
this works for me:
qemu-system-x86_64 -m 2048 -kernel bzImage -daemonize -nographic -vga none -initrd
kata-containers-initrd.img -machine pc,accel=kvm -nographic -no-user-config -nodefaults
-append "initcall_debug console=hvc0 console=hvc1" -device virtio-serial-pci,id=serial0
-device virtconsole,chardev=charconsole0,id=console0 -chardev
socket,id=charconsole0,path=console.sock,server,nowait
sorry that works indeed, no matter what machine is used..
..find that output scrolled so fast... and i should immediately connect the sock after qemu command executing...
Finally , thanks a lot for your help, very nice. especially the Valuable experience...
i will try to make a pr to the documention in the ubuntu section later.
i will try to make a pr to the documention in the ubuntu section later.
great, sounds good
feel free to close this issue when you think that it's no longer needed
Hi @berlinsaint - can this be closed now?