vmware-archive/wardroom

Node tests fail to run

Closed this issue · 1 comments

https://github.com/heptiolabs/wardroom/tree/master/packer#testing-images mentions testing nodes directly via e2e_node.test but there have been upstream changes that seem to have broken this functionality at this time.

I get issues with the kubelet failing to respond to readiness checks even though the cluster otherwise is in a healthy state and passes conformance tests (via sonobuoy).

Errors in the logs look like:

Failure [130.525 seconds]
[BeforeSuite] BeforeSuite
_output/dockerized/go/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:142

  Node 1 disappeared before completing BeforeSuite

  _output/dockerized/go/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:142
------------------------------

And the kubelet itself has lots of warnings:

06:12:48 ip-10-0-21-210 kubelet[2270]: W0126 06:12:48.570608    2270 container.go:409] Failed to create summary reader for "/libcontainer_9469_systemd_test_default.slice": none of the resources are being tracked.
Jan 26 06:12:58 ip-10-0-21-210 kubelet[2270]: W0126 06:12:58.309639    2270 setters.go:144] replacing cloudprovider-reported hostname of ip-10-0-21-210.ec2.internal with overridden hostname of ip-10-0-21-210.ec2.internal
Jan 26 06:12:58 ip-10-0-21-210 kubelet[2270]: W0126 06:12:58.598792    2270 container.go:523] Failed to update stats for container "/libcontainer_9542_systemd_test_default.slice": failed to parse memory.failcnt - open /sys/fs/cgroup/memory/libcontainer_9542_systemd_test_default.slice/memory.failcnt: no such file or directory, continuing to push stats
Jan 26 06:12:58 ip-10-0-21-210 kubelet[2270]: W0126 06:12:58.599096    2270 raw.go:87] Error while processing event ("/sys/fs/cgroup/memory/libcontainer_9542_systemd_test_default.slice": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/libcontainer_9542_systemd_test_default.slice: no such file or directory
Jan 26 06:12:58 ip-10-0-21-210 kubelet[2270]: W0126 06:12:58.599243    2270 raw.go:87] Error while processing event ("/sys/fs/cgroup/devices/libcontainer_9542_systemd_test_default.slice": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/libcontainer_9542_systemd_test_default.slice: no such file or directory

Closed via #134.