Capture pod doesn´t start properly
ferrandinand opened this issue · 5 comments
ferrandinand commented
I cannot start properly the capturer. Seems that sysdig pod doesn´t generate gz because seems that it is not able to start.
Is there any kind of incompatibilty?
Context
Running capture on CoreOS Container Linux by CoreOS 2023.4.0 (Rhyolite)
Nodes on AWS.
Kubernetes version v1.13.4
Command
kubectl capture kube-proxy-z4g6c -ns kube-system -M 10 --snaplen 256
* Setting up /usr/src links from host
ls: cannot access '/host/usr/src': No such file or directory
* Unloading sysdig-probe, if present
* Running dkms install for sysdig
Kernel preparation unnecessary for this kernel. Skipping...
Building module:
cleaning build area.....
make -j4 KERNELRELEASE=4.19.23-coreos-r1 -C /lib/modules/4.19.23-coreos-r1/build M=/var/lib/dkms/sysdig/0.25/build..........(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.19.23-coreos-r1 (x86_64)
Consult /var/lib/dkms/sysdig/0.25/build/make.log for more information.
* Running dkms build failed, dumping /var/lib/dkms/sysdig/0.25/build/make.log
DKMS make.log for sysdig-0.25 for kernel 4.19.23-coreos-r1 (x86_64)
Fri Apr 5 09:40:27 UTC 2019
make: Entering directory '/host/lib/modules/4.19.23-coreos-r1/build'
CC [M] /var/lib/dkms/sysdig/0.25/build/main.o
CC [M] /var/lib/dkms/sysdig/0.25/build/dynamic_params_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/fillers_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/flags_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/ppm_events.o
CC [M] /var/lib/dkms/sysdig/0.25/build/ppm_fillers.o
CC [M] /var/lib/dkms/sysdig/0.25/build/event_table.o
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[3]: *** [../source/scripts/Makefile.build:303: /var/lib/dkms/sysdig/0.25/build/main.o] Error 4
make[3]: *** Deleting file '/var/lib/dkms/sysdig/0.25/build/main.o'
make[3]: *** Waiting for unfinished jobs....
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[3]: *** [../source/scripts/Makefile.build:303: /var/lib/dkms/sysdig/0.25/build/ppm_fillers.o] Error 4
make[3]: *** Deleting file '/var/lib/dkms/sysdig/0.25/build/ppm_fillers.o'
make[3]: *** [../source/scripts/Makefile.build:303: /var/lib/dkms/sysdig/0.25/build/ppm_events.o] Error 4
make[3]: *** Deleting file '/var/lib/dkms/sysdig/0.25/build/ppm_events.o'
make[2]: *** [/host/lib/modules/4.19.23-coreos-r1/source/Makefile:1521: _module_/var/lib/dkms/sysdig/0.25/build] Error 2
make[1]: *** [Makefile:146: sub-make] Error 2
make: *** [Makefile:24: __sub-make] Error 2
make: Leaving directory '/host/lib/modules/4.19.23-coreos-r1/build'
* Trying to load a system sysdig-probe, if present
* Trying to find precompiled sysdig-probe for 4.19.23-coreos-r1
Found kernel config at /proc/config.gz
* Trying to download precompiled module from https://s3.amazonaws.com/download.draios.com/stable/sysdig-probe-binaries/sysdig-probe-0.25-x86_64-4.19.23-coreos-r1-03bf994bd8b87756106f34511fc1aadb.ko
Download failed, consider compiling your own sysdig-probe and loading it or getting in touch with the sysdig community
* Capturing system calls
Unable to load the driver
error opening device /host/dev/sysdig0. Make sure you have root credentials and that the sysdig-probe module is loaded.
----------------------
Event #Calls
----------------------
CommanderJax commented
I encountered the same or a similar error, here is the log, hope it helps;
kc logs -f capture-prometheus-kube-prometheus-0-1554470360
* Setting up /usr/src links from host
* Unloading sysdig-probe, if present
* Running dkms install for sysdig
Kernel preparation unnecessary for this kernel. Skipping...
Building module:
cleaning build area........
make -j56 KERNELRELEASE=4.15.0-44-generic -C /lib/modules/4.15.0-44-generic/build M=/var/lib/dkms/sysdig/0.25/build........................(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.15.0-44-generic (x86_64)
Consult /var/lib/dkms/sysdig/0.25/build/make.log for more information.
* Running dkms build failed, dumping /var/lib/dkms/sysdig/0.25/build/make.log
DKMS make.log for sysdig-0.25 for kernel 4.15.0-44-generic (x86_64)
Fri Apr 5 13:19:45 UTC 2019
make: Entering directory '/host/usr/src/linux-headers-4.15.0-44-generic'
CC [M] /var/lib/dkms/sysdig/0.25/build/main.o
CC [M] /var/lib/dkms/sysdig/0.25/build/dynamic_params_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/fillers_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/flags_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/ppm_fillers.o
CC [M] /var/lib/dkms/sysdig/0.25/build/ppm_events.o
CC [M] /var/lib/dkms/sysdig/0.25/build/event_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/syscall_table.o
CC [M] /var/lib/dkms/sysdig/0.25/build/ppm_cputime.o
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[1]: *** [scripts/Makefile.build:332: /var/lib/dkms/sysdig/0.25/build/fillers_table.o] Error 4
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [scripts/Makefile.build:332: /var/lib/dkms/sysdig/0.25/build/syscall_table.o] Error 4
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[1]: *** [scripts/Makefile.build:332: /var/lib/dkms/sysdig/0.25/build/ppm_fillers.o] Error 4
gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[1]: *** [scripts/Makefile.build:332: /var/lib/dkms/sysdig/0.25/build/ppm_events.o] Error 4
make: *** [Makefile:1551: _module_/var/lib/dkms/sysdig/0.25/build] Error 2
make: Leaving directory '/host/usr/src/linux-headers-4.15.0-44-generic'
* Trying to load a system sysdig-probe, if present
* Trying to find precompiled sysdig-probe for 4.15.0-44-generic
Found kernel config at /host/boot/config-4.15.0-44-generic
* Trying to download precompiled module from https://s3.amazonaws.com/download.draios.com/stable/sysdig-probe-binaries/sysdig-probe-0.25-x86_64-4.15.0-44-generic-a298fcd69ad192c225fe536a5c6ad115.ko
Download succeeded, loading module
* Capturing system calls
Unable to load the driver
error mapping the ring buffer for device /host/dev/sysdig51
----------------------
Event #Calls
----------------------
nestorsalceda commented
Woah, what a detailed description!
Right now, the capture plugin only supports the old kernel module loading. This is not well supported for CoreOS, it needs the eBPF probe. Next feature I'm planning to do is to enable the eBPF support.
I will keep you updated.
ferrandinand commented
Great, I am looking forward seeing it in action.
Keep me updated
nestorsalceda commented
I just merged the eBPF support for kubectl capture.
You will need to run the command using --ebpf flag:
kubectl capture kube-proxy-z4g6c -ns kube-system --ebpf -M 10 --snaplen 256
Please, give a try @ferrandinand and thanks for reporting :)
ferrandinand commented
Working like a charm.
So great seing it in action with sysdig-inspect.