Attaching EBPF program returns no such file or directory
Manxiaxia opened this issue · 5 comments
Hi Florian,
I am currently working on a project which requires to load ebpf program to the tc ingress hook. I followed the example you provided a while ago: https://gist.github.com/florianl/8f421e57f419fa9a50eb5b085363de66. There are some changes on the go-tc package but I tried to update the sample code to adapt the newest version.
The error I got is: could not assign eBPF: netlink receive: no such file or directory.
Brief introduction on what I did:
bpf program trying to load:
/*
#cgo CFLAGS: -I/usr/include/bcc/compat
#cgo LDFLAGS: -lbcc
#include <bcc/bcc_common.h>
#include <bcc/libbpf.h>
void perf_reader_free(void *ptr);
*/
import "C"
const source string = `
#define KBUILD_MODNAME "tc_eBPF"
#include <uapi/linux/bpf.h>
int tc_eBPF(struct __sk_buff *skb) {
bpf_trace_printk("hello world\n");
return 0;
}
`
then using the gobpf library to load bpf program
module := bpf.NewModule(source, []string{"-w"})
defer module.Close()
fn, err := module.Load("tc_eBPF", C.BPF_PROG_TYPE_SCHED_CLS, 1, 65536)
if err != nil {
fmt.Fprintf(os.Stderr, "failed to load ebpf prog: %v\n", err)
return
}
after that, open the rtnl and find the deviceID
rtnl, err := tc.Open(&tc.Config{})
if err != nil {
fmt.Fprintf(os.Stderr, "could not open rtnetlink socket: %v\n", err)
return
}
defer func() {
if err := rtnl.Close(); err != nil {
fmt.Fprintf(os.Stderr, "could not close rtnetlink socket: %v\n", err)
}
}()
devID, err := net.InterfaceByName("wlo1")
if err != nil {
fmt.Fprintf(os.Stderr, "could not get interface ID: %v\n", err)
return
}
once that part is covered, I added clsact qdisc for adding ebpf filter
qdisc := tc.Object{
tc.Msg{
Family: unix.AF_UNSPEC,
Ifindex: uint32(devID.Index),
Handle: core.BuildHandle(tc.HandleIngress, 0x0000),
Parent: tc.HandleIngress,
Info: 0,
},
tc.Attribute{
Kind: "clsact",
},
}
//add clsact qdisc to tc for attaching ebpf
if err := rtnl.Qdisc().Add(&qdisc); err != nil {
fmt.Fprintf(os.Stderr, "could not assign clsact to lo: %v\n", err)
return
}
defer deleteAddedQdisc(rtnl, qdisc)
the delete method provided in sample code seems not working, so I modified it a little bit for testing purpose:
func deleteAddedQdisc(rtnl *tc.Tc, addedQdisc tc.Object) {
qdiscs, err := rtnl.Qdisc().Get()
if err != nil {
fmt.Println(err.Error())
return
}
for _, qdisc := range qdiscs {
if qdisc.Kind == addedQdisc.Kind && qdisc.Ifindex == addedQdisc.Ifindex {
err := rtnl.Qdisc().Delete(&qdisc)
if err != nil {
fmt.Printf(err.Error())
return
}
}
}
}
at the end, I created a tc bpf filter
bpfFn := uint32(fn)
bpfFlag := uint32(0x1)
//
filter := tc.Object{
Msg: tc.Msg{
Family: unix.AF_UNSPEC,
Ifindex: uint32(devID.Index),
Handle: core.BuildHandle(tc.HandleIngress, 0x0000),
Parent: tc.HandleIngress,
Info: 0x300,
},
Attribute: tc.Attribute{
Kind: "bpf",
BPF: &tc.Bpf{
FD: &bpfFn,
Flags: &bpfFlag,
},
},
}
if err := rtnl.Filter().Add(&filter); err != nil {
fmt.Fprintf(os.Stderr, "could not assign eBPF: %v\n", err)
return
}
That's pretty much it, I am also gonna to include a full program at the end. After run the program, I got could not assign ebpf: netlink receive: no such file or directory error. My assumption is that when adding the bpf filter, it is trying to find the ebpf file but cannot find it. I am wondering if I did something wrong? Thank you.
package main
import (
"fmt"
"github.com/florianl/go-tc"
"github.com/florianl/go-tc/core"
bpf "github.com/iovisor/gobpf/bcc"
"golang.org/x/sys/unix"
"net"
"os"
)
/*
#cgo CFLAGS: -I/usr/include/bcc/compat
#cgo LDFLAGS: -lbcc
#include <bcc/bcc_common.h>
#include <bcc/libbpf.h>
void perf_reader_free(void *ptr);
*/
import "C"
const source string = `
#define KBUILD_MODNAME "tc_eBPF"
#include <uapi/linux/bpf.h>
int tc_eBPF(struct __sk_buff *skb) {
bpf_trace_printk("hello world\n");
return 0;
}
`
func main() {
module := bpf.NewModule(source, []string{"-w"})
defer module.Close()
fn, err := module.Load("tc_eBPF", C.BPF_PROG_TYPE_SCHED_CLS, 1, 65536)
if err != nil {
fmt.Fprintf(os.Stderr, "failed to load ebpf prog: %v\n", err)
return
}
rtnl, err := tc.Open(&tc.Config{})
if err != nil {
fmt.Fprintf(os.Stderr, "could not open rtnetlink socket: %v\n", err)
return
}
defer func() {
if err := rtnl.Close(); err != nil {
fmt.Fprintf(os.Stderr, "could not close rtnetlink socket: %v\n", err)
}
}()
devID, err := net.InterfaceByName("wlo1")
if err != nil {
fmt.Fprintf(os.Stderr, "could not get interface ID: %v\n", err)
return
}
qdisc := tc.Object{
tc.Msg{
Family: unix.AF_UNSPEC,
Ifindex: uint32(devID.Index),
Handle: core.BuildHandle(tc.HandleIngress, 0x0000),
Parent: tc.HandleIngress,
Info: 0,
},
tc.Attribute{
Kind: "clsact",
},
}
//add clsact qdisc to tc for attaching ebpf
if err := rtnl.Qdisc().Add(&qdisc); err != nil {
fmt.Fprintf(os.Stderr, "could not assign clsact to lo: %v\n", err)
return
}
defer deleteAddedQdisc(rtnl, qdisc)
//programName := "tc_eBPF"
bpfFn := uint32(fn)
bpfFlag := uint32(0x1)
//
filter := tc.Object{
Msg: tc.Msg{
Family: unix.AF_UNSPEC,
Ifindex: uint32(devID.Index),
Handle: core.BuildHandle(tc.HandleIngress, 0x0000),
Parent: tc.HandleIngress,
Info: 0x300,
},
Attribute: tc.Attribute{
Kind: "bpf",
BPF: &tc.Bpf{
FD: &bpfFn,
Flags: &bpfFlag,
},
},
}
if err := rtnl.Filter().Add(&filter); err != nil {
fmt.Fprintf(os.Stderr, "could not assign eBPF: %v\n", err)
return
}
//time.Sleep(30*time.Hour)
// cat sys/kernel/debug/tracing/trace_pipe
}
func deleteAddedQdisc(rtnl *tc.Tc, addedQdisc tc.Object) {
qdiscs, err := rtnl.Qdisc().Get()
if err != nil {
fmt.Println(err.Error())
return
}
for _, qdisc := range qdiscs {
if qdisc.Kind == addedQdisc.Kind && qdisc.Ifindex == addedQdisc.Ifindex {
err := rtnl.Qdisc().Delete(&qdisc)
if err != nil {
fmt.Printf(err.Error())
return
}
}
}
}
Thanks for your report 👍
The feedback from the netlink subsystem of the kernel is limited. Reading your code I noticed some differences to the gist example.
filter.Attribute.BPF.Nameis not setfilter.Msg.Handleshould not be the same asqdisc.Msg.Handle
Wrt deleting the filer - there is already issue #17 . But I didn't find time taking a closer look so far.
Hi Florian,
Thank you for your response. I am actually a bit confused on the filter.Attribute.BPF.Name filed. The name used on the gist example is tc_prog but the name of the ebpf program is tc_eBPF. I am wondering what does the Name file represnets?
I actually tried to run the program with the filter.Attribute.BPF.Name field set to name of the ebpf program, but still got the same error. Also, the filter.Msg.Handle is changed to 0.
I just remember that filter.Attribute.BPF.Name can be set to something arbitrary.
The following settings I applied to your example and it works now:
qdisc := tc.Object{
tc.Msg{
Family: unix.AF_UNSPEC,
Ifindex: devIfaceIndex,
Handle: core.BuildHandle(0xFFFF, 0x0000),
Parent: tc.HandleIngress,
},
tc.Attribute{
Kind: "clsact",
},
}
[...]
filter := tc.Object{
Msg: tc.Msg{
Family: unix.AF_UNSPEC,
Ifindex: devIfaceIndex,
Handle: 0,
Parent: 0xFFFFFFF2,
Info: 0x10300,
},
Attribute: tc.Attribute{
Kind: "bpf",
BPF: &tc.Bpf{
FD: &bpfFn,
Flags: &bpfFlag,
},
},
}
Hi Florian,
I noticed that the changes you made are filter.Msg.Parent and filter.Msg.Info.
Based on my understanding, since the parent of the qdisc is tc.HandleIngress which is 0xFFFFFFF1, therefore, the ID of the clsact qdisc we created will be 0xFFFFFFF2. Since we want to assign the filter to clsact qdisc, the parent of the filter will then be 0xFFFFFFF2. Is that right?
In terms of filter.Msg.Info filed, I am wondering is that field mandatory? I am not quite sure understand the meaning of that field. I tried to look through the tc man page but I got no luck.
Thank you so much for your help, I really appreciate it.
Besides the man page tc(8) there is more documentation at https://www.infradead.org/~tgr/libnl/doc/route.html#route_tc.
Therefore I'm closing this issue.