cuviper/probe-rs

How do I know that its working?

Closed this issue · 5 comments

Fiedzia commented

After compiling simple example (adding thread::sleep to the loop to be able to observe it) I am getting:

/snap/bcc/current/usr/bin/python /snap/bcc/8/usr/share/bcc/tools/tplist -p pidof testprobes -v 'foo*'
/home/maciej/tmp/probes/src/target/debug/testprobes foo:begin [sema 0x0]
1 location(s)
0 argument(s)
/home/maciej/tmp/probes/src/target/debug/testprobes foo:end [sema 0x0]
1 location(s)
0 argument(s)
/home/maciej/tmp/probes/src/target/debug/testprobes foo:loop [sema 0x0]
1 location(s)
2 argument(s)

so this seem to work

bcc.trace -p pidof testprobes 'u::loop'
PID TID COMM FUNC
31292 31292 testprobes loop
31292 31292 testprobes loop
...

so events are reported, but when I am trying to look at parameters:

bcc.trace -p pidof testprobes 'u::loop "%d", arg1'
check dmesg output for possible cause
Failed to attach BPF to kprobe

dmesg shows:

[231732.373470] Failed to allocate trace_probe.(-22)

cuviper commented

I haven't really used bcc, but you can use readelf -n to look at the ELF notes and see if it looks reasonable. That what the in-tree tests/readelf.rs is doing on itself. That output will look something like:

Displaying notes found at file offset 0x000b75dc with length 0x00000074:
  Owner                 Data size       Description
  stapsdt              0x00000022       NT_STAPSDT (SystemTap probe descriptors)
    Provider: test
    Name: foo
    Location: 0x00000000000177e3, Base: 0x0000000000097130, Semaphore: 0x0000000000000000
    Arguments:
  stapsdt              0x00000028       NT_STAPSDT (SystemTap probe descriptors)
    Provider: test
    Name: bar
    Location: 0x00000000000177e4, Base: 0x0000000000097130, Semaphore: 0x0000000000000000
    Arguments: -8@$42

Failed to attach BPF to kprobe

It should be a uprobe, not kprobe -- is this just a bcc or bpf oddity?

[231732.373470] Failed to allocate trace_probe.(-22)

FWIW, -22 is EINVAL, "Invalid argument". Perhaps your "%d" doesn't match what bcc wants for that arg1? Note that this library casts all arguments to i64, because I don't know any way to do proper type resolution within this sort of macro. So maybe use "%ld" or something like that?

Fiedzia commented

It should be a uprobe, not kprobe -- is this just a bcc or bpf oddity?

Don't know. Maybe its me not understanding how it works and what I am doing (most likely).

So maybe use "%ld" or something like that?

No luck so far.

I've set up a repo with my code in description of what I am doing: https://github.com/Fiedzia/rust-bpf-test
Could you take a look and see if it works at all?

Fiedzia commented

One more thing - looking at the source I've found out that I can use gdb to set breakpoints on probes - and that works fine too. So it all comes to me not knowing how to use bcc probably.

cuviper commented

This is what I see for your binary:

Displaying notes found at file offset 0x0004c49c with length 0x000000c4:
  Owner                 Data size       Description
  stapsdt              0x00000023       NT_STAPSDT (SystemTap probe descriptors)
    Provider: foo
    Name: begin
    Location: 0x0000000000005eeb, Base: 0x000000000003dcb1, Semaphore: 0x0000000000000000
    Arguments:
  stapsdt              0x0000003d       NT_STAPSDT (SystemTap probe descriptors)
    Provider: foo
    Name: loop
    Location: 0x0000000000006092, Base: 0x000000000003dcb1, Semaphore: 0x0000000000000000
    Arguments: -8@-456(%rbp) -8@-464(%rbp)
  stapsdt              0x00000021       NT_STAPSDT (SystemTap probe descriptors)
    Provider: foo
    Name: end
    Location: 0x00000000000061d7, Base: 0x000000000003dcb1, Semaphore: 0x0000000000000000
    Arguments:

So loop does have two arguments, both at offsets on the stack. And I can get gdb working with this, even printing the arguments correctly, which it calls $_probe_arg0 and $_probe_arg1.

Actually, I got bcc working too, with just 'u::loop "%d", arg1' as you had before. Maybe your kernel is missing some useful options? You said you have Ubuntu 16.10, which should be plenty new enough.

Can you try some probes that are not Rust-based? Then we can completely rule out it being a problem with this crate. Thanks!

Fiedzia commented

I've installed bcc from ubuntu repository instead of using snap and that solved the problem.

  • 👍1