netlink receive: no such file or directory on Android 12

Question

netlink receive: no such file or directory on Android 12

kittyzero520 opened this issue a year ago · 27 comments

OS: [安卓12]
Arch: [e.g. arm_aarch64]
Kernel Version: [ 5.10.110]
Version: [ecapture-v0.5.1-android-aarch64.tar.gz]
使用ecapture-v0.5.1-android-aarch64.tar.gz在安卓12上运行命令报错，报错信息，如下：

ecapt tls -w save_android.pcapng -i eth0

module run failed, [skip it]. error:couldn't start bootstrap manager error:2 errors occurred:
* error:error:netlink receive: no such file or directory , couldn't add a ", err clsact" qdisc to interface 5, {UID:, EbpfFuncName:egress_cls_func}
* error:error:netlink receive: no such file or directory , couldn't add a ", err clsact" qdisc to interface 5, {UID:, EbpfFuncName:ingress_cls_func}

, probes activation validation failed .
tls_2023/04/14 01:45:40 ECAPTURE :: No runnable modules, Exit(1)

ecapt tls -w save_android.pcapng -i wlan0
module run failed, [skip it]. error:route ip+net: no such network interface
tls_2023/04/14 01:48:57 ECAPTURE :: No runnable modules, Exit(1)

Answer 1 · 2023-04-15T14:48:39.000Z

similar #331

Answer 2 · 2023-04-16T04:01:30.000Z

在 #331 里也提到了这个报错，按照他的环境，我无法重现。

你可以自己先多测试测试，尝试给出其他更多环境不同的信息吗？

发一下 tc qdisc add dev eth0 clsact的结果

In #331, this error was also mentioned. According to their environment, I am unable to reproduce it.

Can you please do more testing yourself and try to provide additional information about different environments?

upload result please ，shell : tc qdisc add dev eth0 clsact

bin/ecapture tls -i eth0 -w a.pcapng
tls_2023/04/16 03:59:22 ECAPTURE :: ecapture Version : linux_x86_64:0.5.1-20230415-fffcd0f:[CORE]
tls_2023/04/16 03:59:22 ECAPTURE :: Pid Info : 9095
tls_2023/04/16 03:59:22 ECAPTURE :: Kernel Info : 6.2.8
2023/04/16 03:59:22 read keylogger :/etc/ld.so.conf.d/*.conf error .
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	module initialization
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	Module.Run()
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	TC MODEL
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	OpenSSL/BoringSSL version not found from shared library file, used default version:linux_default_3_0
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	HOOK type:2, binrayPath:/lib/libssl.so.3
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	Ifname:eth0, Ifindex:2,  Port:443, Pcapng filepath:/root/ecapture/a.pcapng
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	Hook masterKey function:SSL_write
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	target all process.
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	target all users.
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	BPF bytecode filename:user/bytecode/openssl_3_0_0_kern.o
tls_2023/04/16 03:59:24 EBPFProbeOPENSSL	module started successfully.
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	module initialization
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	Module.Run()
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	BPF bytecode filename:user/bytecode/gnutls_kern.o
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	HOOK type:2, binrayPath:/usr/lib/libgnutls.so.30
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	target all process.
tls_2023/04/16 03:59:25 EBPFProbeGNUTLS	module started successfully.
tls_2023/04/16 03:59:25 EBPFProbeNSPR	module initialization failed. [skip it]. error:stat /usr/lib/libnspr4.so: no such file or directory
tls_2023/04/16 03:59:25 ECAPTURE :: 	cant found module EBPFProbeGoTLS config info.
tls_2023/04/16 03:59:25 ECAPTURE :: 	start 2 modules

Answer 3 · 2023-04-17T02:50:54.000Z

blueline:/ # tc qdisc add dev eth0 clsact
RTNETLINK answers: No such file or directory

Answer 4 · 2023-04-17T03:03:52.000Z

歪个楼，请问你的blueline是怎么用上5.10的内核的，可以展开一下吗 😃

Answer 5 · 2023-04-17T03:17:09.000Z

17万刀片服务器自带的

Answer 6 · 2023-04-17T03:19:20.000Z

了解了，那应该是 redroid 或者 cuttlefish 之类的技术吧?

Answer 7 · 2023-04-17T13:15:28.000Z

blueline:/ # tc qdisc add dev eth0 clsact RTNETLINK answers: No such file or directory

应该是你的内核不支持network emulation。内核编译需要启用相关配置，你可以参考如下链接。

It should be that your kernel does not support network emulation. Enabling relevant configurations is required during kernel compilation, and you can refer to the following link.

https://itecnote.com/tecnote/linux-rtnetlink-answers-no-such-file-or-directory-error/
https://cateee.net/lkddb/web-lkddb/NET_SCH_NETEM.html
https://itecnote.com/tecnote/linux-rtnetlink-answers-no-such-file-or-directory-error/

Answer 8 · 2023-04-21T09:42:59.000Z

重新编译内容后，发现抓到的pcap中没什么请求内容

Answer 9 · 2023-04-21T09:44:42.000Z

抓包文件
save_pcap.pcapng.zip

Answer 10 · 2023-04-21T14:32:36.000Z

你使用的启动命令、测试shell分别是什么？我觉得大概是你监听的网卡不对，不是流量经过的网卡

What are the startup commands and test shells you are using? I think it's probably because you're monitoring the wrong network interface, not the one that the traffic is passing through.

Answer 11 · 2023-04-22T05:50:52.000Z

我想把抓包的数据写到数据库，将记录保存下来

Answer 12 · 2023-04-23T05:49:16.000Z

应该是eth0

130|blueline:/data/local/tmp # ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:13 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:600 TX bytes:600

eth0 Link encap:Ethernet HWaddr 48:ad:08:45:1c:01
inet addr:192.168.127.152 Bcast:192.168.127.255 Mask:255.255.255.0
inet6 addr: fe80::1b60:bd0c:a877:bcaf/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:92078896 errors:0 dropped:1213 overruns:0 frame:0
TX packets:54925595 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:157061853928 TX bytes:4915228285

Answer 13 · 2023-04-23T05:50:00.000Z

使用命令 tc qdisc add dev eth0 clsact

Answer 14 · 2023-04-23T13:11:24.000Z

使用命令 tc qdisc add dev eth0 clsact

嗯？这条命令有什么问题吗？

你这个问题的错误原因，在 #347 (comment) 描述的很详细了。

Answer 15 · 2023-04-23T13:57:47.000Z

执行命令不反馈空记录了，到记录无法写入pcap

Answer 16 · 2023-04-23T14:12:02.000Z

_2023/04/23 06:51:44 EBPFProbeOPENSSL saving pcapng file /data/local/tmp/test.pcapng
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save pcanNP failed, error:Can't send statistics for non existent interface 8; have only 3 interfaces.
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save 1 packets into pcapng file.
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close.
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close

Answer 17 · 2023-04-23T14:12:52.000Z

文件都1kb大小

Answer 18 · 2023-04-23T14:28:21.000Z

_2023/04/23 06:51:44 EBPFProbeOPENSSL saving pcapng file /data/local/tmp/test.pcapng

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save pcanNP failed, error:Can't send statistics for non existent interface 8; have only 3 interfaces.

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save 1 packets into pcapng file.

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close.

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close

完整的命令行发出来，别总发不全的信息。沟通成本很高

Answer 19 · 2023-04-23T14:49:56.000Z

blueline:/data/local/tmp # ./ecapt tls -i eth0 -w test.pcapng
tls_2023/04/23 14:37:54 ECAPTURE :: ecapture Version : androidgki_aarch64:0.5.1-20230408-e1afbb8:[CORE]
tls_2023/04/23 14:37:54 ECAPTURE :: Pid Info : 24515
tls_2023/04/23 14:37:54 ECAPTURE :: Kernel Info : 5.10.110
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL module initialization
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL Module.Run()
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL TC MODEL
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL OpenSSL/BoringSSL version not found, used default version :android_default
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL HOOK type:2, binrayPath:/apex/com.android.conscrypt/lib64/libssl.so
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL Ifname:eth0, Ifindex:8, Port:443, Pcapng filepath:/data/local/tmp/test.pcapng
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL Hook masterKey function:SSL_in_init
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL target all process.
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL target all users.
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL BPF bytecode filename:user/bytecode/boringssl_1_1_1_kern.o
tls_2023/04/23 14:37:56 EBPFProbeOPENSSL module started successfully.
tls_2023/04/23 14:37:56 ECAPTURE :: start 1 modules
tls_2023/04/23 14:38:11 TLS1_3_VERSION: save CLIENT_RANDOM c819e15dbd9b64b9583a643c9fee242d03219e97eadaf18d29f6ee8525aa1448 to file success, 778 bytes
tls_2023/04/23 14:39:00 TLS1_3_VERSION: save CLIENT_RANDOM 3c9fe3beda23815ce1e7d85002324f17dacff3de1ec92d591f4cc14aae836e64 to file success, 778 bytes

^Ctls_2023/04/23 14:39:16 EBPFProbeOPENSSL saving pcapng file /data/local/tmp/test.pcapng
tls_2023/04/23 14:39:16 EBPFProbeOPENSSL save pcanNP failed, error:Can't send statistics for non existent interface 8; have only 3 interfaces.
tls_2023/04/23 14:39:16 EBPFProbeOPENSSL save 1 packets into pcapng file.
tls_2023/04/23 14:39:16 EBPFProbeOPENSSL close.
tls_2023/04/23 14:39:17 EBPFProbeOPENSSL

Answer 20 · 2023-04-23T14:59:47.000Z

Can't send statistics for non existent interface 8; have only 3 interfaces.

这里抱错了，我第一次遇到，租需要debug一下。你的运行环境可以给一下吗？详细信息。

Answer 21 · 2023-04-23T15:10:51.000Z

可以的，给个邮箱我发你

Answer 22 · 2023-04-23T16:38:18.000Z

版本信息贴在这呗，不用发邮箱

Answer 23 · 2023-04-23T17:32:00.000Z

OS: [安卓12]
Arch: [e.g. arm_aarch64]
Kernel Version: [ 5.10.110]
Version: [ecapture-v0.5.1-android-aarch64.tar.gz]
硬件设备：刀片服务器

这是环境信息，我的意思你要不远程设备看下

Answer 24 · 2023-04-24T01:32:19.000Z

嗯，这硬件配置，我确实准备不来。你在我微信公众号里留言吧，我到时加你。

Answer 25 · 2023-04-24T02:18:59.000Z

感谢大神的支持

Answer 26 · 2023-04-24T03:25:10.000Z

已关注公众号，todesk远程信息已发

Answer 27 · 2023-04-28T11:56:08.000Z

blueline:/data/local/tmp # ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
8: eth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 48:ad:08:45:1c:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0

eth0网卡ID是8，实际上一共就2个网卡。在gopacket包里，判断不对，报错了。

// pcapgo/ngwrite.go
func (w *NgWriter) WritePacket(ci gopacket.CaptureInfo, data []byte) error {
	if ci.InterfaceIndex >= int(w.intf) || ci.InterfaceIndex < 0 {
		return fmt.Errorf("Can't send statistics for non existent interface %d; have only %d interfaces", ci.InterfaceIndex, w.intf)
	}
// ...