Documentation for enabling CONFIG_FTRACE_SYSCALLS and disabling CONFIG_TRACEFS_DISABLE_AUTOMOUNT
droidresearch opened this issue · 22 comments
The documentation for getting started with android has the following instruction to make the pulsar available for android
enable CONFIG_FTRACE_SYSCALLS and CONFIG_BPF_LSM
disable CONFIG_TRACEFS_DISABLE_AUTOMOUNT
However, starting from android 13, editing the .config file to enable/disable the config options is changed as there is a bazel based build from android-13 kernels.
Could you please point to any references on how to build the kernel with these two options enabled.
Thank you
For a custom kernel for Android suitable for Pulsar, we referred to this page: https://source.android.com/docs/setup/build/building-kernels#customize-config.
If you intend to build the kernel with Bazel, you can refer to this page: https://android.googlesource.com/kernel/build/+/refs/heads/master/kleaf/README.md.
However, we have not yet tried this procedure.
Please let us know if you encounter any further issues.
Thanks for the response. Yes, I have been trying to figure out similar options to cusomize kernel config using bazel build. As, I am not able to make the device boot if the kernel build is customized with the traditional build config method.
Could you please let me know if you used emulator or a real device for testing the custom kernel and did you use fastboot to flash the custom kernel
Sorry accidentally closed this
We utilized both the emulator (x86_64 and aarch64) and real devices (aarch64 only) with Lineage, following the device-specific guides (https://wiki.lineageos.org/devices/).
Thanks, I will check with LineageOS.
I got the rest of the modules running except file monitor. the following is the error message I see in the logs. Could you help with what symbol did I miss?
ERROR pulsar::pulsard::module_manager] Error in module file-system-monitor. Module stopped. ProgramLoadError { program: "lsm path_mknod", program_error: Btf(UnknownBtfTypeName { type_name: "bpf_lsm_path_mknod" }) }
What platform are you using (emulator and real device) and what architecture (x86_64 or aarch64)?
I have compiled a custom kernel for emulator with all the config options for emulator x86_64
vsoc_x86_64:/ # uname -a
Linux localhost 5.15.104-android13-8-00025-g8dd49c100504-dirty #1 SMP PREEMPT Mon Jul 17 19:36:51 UTC 2023 x86_64 Toybox
vsoc_x86_64:/ #
Make sure that CONFIG_BPF_LSM is enabled, and check that the output of cat /sys/kernel/security/lsm contains the string "bpf". If it does not, BPF LSM must be enabled, for example, either in Grub or directly in the kernel at compile-time.
Thanks, It is strange that I see that in /proc/config.gz (CONFIG_BPF_LSM=y) is set but do not see /sys/kernel/security/lsm
#
# BPF subsystem
#
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_BPF_JIT_DEFAULT_ON=y
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
# CONFIG_BPF_PRELOAD is not set
CONFIG_BPF_LSM=y
# end of BPF subsystem
My bad! there securityfs was not mounted. Now, mount point is established. and I do see the lsm in /sys/kernel/security
root@localhost:/data/local/tmp# ls /sys/kernel/security/
integrity lsm
root@localhost:/data/local/tmp# ./pulsar-exec-x86_64-static pulsard --config-file=./pulsar.ini
[2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module process-monitor
[2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module file-system-monitor
[2023-07-19T10:46:42Z ERROR pulsar::pulsard::module_manager] Error in module file-system-monitor. Module stopped. ProgramLoadError { program: "lsm path_mknod", program_error: Btf(UnknownBtfTypeName { type_name: "bpf_lsm_path_mknod" }) }
[2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module network-monitor
[2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module logger
[2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module rules-engine
[2023-07-19T10:46:42Z INFO pulsar::pulsard::daemon] Starting module desktop-notifier
You should see the content of the file /sys/kernel/security/lsm
for example, on my system it is:
$ cat /sys/kernel/security/lsm
capability,landlock,lockdown,yama,bpf
Please check whether this file contains the string "bpf".
Ok. In my system, I see bpf but landlock, lockdown, yama is missing!! Please see the output below.
cat /sys/kernel/security/lsm capability,selinux,bpf
Your emulator seems to be fine with the BPF LSM configurations.
Check if debugfs is mounted. If it isn't, run
mount -t debugfs none /sys/kernel/debug
Yes, the debugfs also is mounted and has the following entries
acpi block cleancache devfreq dmaengine extfrag hid kfence lru_gen_full regmap slab swiotlb virtio-ports
asoc bluetooth clear_warn_once device_component dri f2fs ieee80211 kprobes opp regulator sleep_time tracing wakeup_sources
bdi btt clk devices_deferred dynamic_debug fault_around_bytes iio kvm ramdisk_pages remoteproc split_huge_pages ufshcd x86
binder cec cma dma_buf error_injection gpio interconnect lru_gen ras sched suspend_stats usb
vsoc_x86_64:/data/local/tmp #
ok, try also to mount tracefs, because it is checked before /sys/kernel/debug/tracing:
mount -t tracefs tracefs /sys/kernel/tracing
Ok. Please find the output of the listing the tracefs mounted to /sys/kernel/tracing
ls -lhrt /sys/kernel/tracing/
total 0
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_event_pid
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_event_notrace_pid
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 set_event
-r--r--r--. 1 root 3012 0 Jul 19 10:54 available_events
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 tracing_thresh
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 tracing_max_latency
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 tracing_cpumask
-r--r--r--. 1 root 3012 0 Jul 19 10:54 trace_pipe
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 trace_options
--w--w----. 1 root 3012 0 Jul 19 10:54 trace_marker_raw
--w--w--w-. 1 root 3012 0 Jul 19 10:54 trace_marker
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 trace_clock
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 trace
-r--r--r--. 1 root 3012 0 Jul 19 10:54 timestamp_mode
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 snapshot
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_graph_notrace
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_graph_function
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_ftrace_pid
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_ftrace_notrace_pid
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_ftrace_notrace
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 set_ftrace_filter
-r--r--r--. 1 root 3012 0 Jul 19 10:54 saved_tgids
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 saved_cmdlines_size
-r--r--r--. 1 root 3012 0 Jul 19 10:54 saved_cmdlines
drwxr-xr-x. 4 root 3012 0 Jul 19 10:54 per_cpu
drwxr-xr-x. 2 root 3012 0 Jul 19 10:54 options
--w-------. 1 root 3012 0 Jul 19 10:54 free_buffer
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 error_log
-r--r--r--. 1 root 3012 0 Jul 19 10:54 enabled_functions
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 current_tracer
-r--r--r--. 1 root 3012 0 Jul 19 10:54 buffer_total_size_kb
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 buffer_size_kb
-r--r--r--. 1 root 3012 0 Jul 19 10:54 buffer_percent
-r--r--r--. 1 root 3012 0 Jul 19 10:54 available_tracers
-r--r--r--. 1 root 3012 0 Jul 19 10:54 available_filter_functions
-r--r--r--. 1 root 3012 0 Jul 19 10:54 README
-r--r--r--. 1 root 3012 0 Jul 19 10:54 uprobe_profile
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 uprobe_events
-r--r--r--. 1 root 3012 0 Jul 19 10:54 printk_formats
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 max_graph_depth
-r--r--r--. 1 root 3012 0 Jul 19 10:54 kprobe_profile
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 kprobe_events
drwxr-xr-x. 4 root 3012 0 Jul 19 10:54 instances
-rw-r--r--. 1 root 3012 0 Jul 19 10:54 dynamic_events
-r--r--r--. 1 root 3012 0 Jul 19 10:54 dyn_ftrace_total_info
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 tracing_on
-rw-rw-rw-. 1 root 3012 0 Jul 19 10:54 synthetic_events
drwxr-xr-x. 118 root 3012 0 Jul 19 10:54 events
The emulator appears to be well configured.
I will try to recreate the issue as soon as possible and I will let you know.
Thank you so much for the advice so far.
I managed to fix the api_socket_path issue and got the modules running except filesystem monitor.
| MODULE | VERSION | STATUS |
+==============================================================================+
| desktop-notifier | 0.6.0 | Running |
|---------------------+---------+----------------------------------------------|
| file-system-monitor | 0.6.0 | Failed("failed program load lsm path_mknod") |
|---------------------+---------+----------------------------------------------|
| logger | 0.6.0 | Running |
|---------------------+---------+----------------------------------------------|
| network-monitor | 0.6.0 | Running |
|---------------------+---------+----------------------------------------------|
| process-monitor | 0.6.0 | Running |
|---------------------+---------+----------------------------------------------|
| rules-engine | 0.6.0 | Running |
+---------------------+---------+----------------------------------------------+
Ok. It looks like there is a regression in some version. I am testing one by one. The first version works fine (atleast the status of all modules are green but there is no monitor command so no logs)
./pulsar-exec-static pulsar status
+---------------------+---------+---------+
| MODULE | VERSION | STATUS |
+=========================================+
| file-system-monitor | 0.0.1 | Running |
|---------------------+---------+---------|
| logger | 0.0.1 | Running |
|---------------------+---------+---------|
| network-monitor | 0.0.1 | Running |
|---------------------+---------+---------|
| process-monitor | 0.0.1 | Running |
|---------------------+---------+---------|
| rules-engine | 0.0.1 | Running |
|---------------------+---------+---------|
| syscall-monitor | 0.0.1 | Running |
+---------------------+---------+---------+
I have a working environment with the following config fix to kernel
CONFIG_SECURITY_LANDLOCK=y CONFIG_SECURITY_LOCKDOWN_LSM=y CONFIG_SECURITY_YAMA=y
Thank you so much for the inputs.
The emulator appears to be well configured. I will try to recreate the issue as soon as possible and I will let you know.