Socketcan candump hardware timestamp usage
tuna-f1sh opened this issue · 19 comments
Does the firmware support the candump -H hardware timestamp flag? Quickly looking at the firmware, a timestamp is provided in the gsusb frame using the TIM2 but I get zeros from candump can0 -H -t a:
(0000000000.000000) can0 500 [8] 03 5A 00 5A 23 00 64 05
(0000000000.000000) can0 550 [3] 54 05 00
(0000000000.000000) can0 501 [8] 03 5A 00 5A 23 00 64 16
(0000000000.000000) can0 532 [3] F5 06 00
The same flag works with a PCAN USB device so not sure whether this is supported by candleLight or I am doing something wrong?
The Linux driver doesn't support timestamps, yet, but this can be added. But I don't know about the status of the firmware.
@marckleinebudde ah yes I see that gs_usb doesn't have the timestamp_us in the frame: https://github.com/torvalds/linux/blob/master/drivers/net/can/usb/gs_usb.c#L216 but the firmware does https://github.com/candle-usb/candleLight_fw/blob/master/include/gs_usb.h#L256
So since the kernel seems to support this it would be relatively easy to add to the gs_usb module?
hopefully :)
Create a new struct:
struct classic_can_ts {
u8 data[8];
u32 timestamp_us;
} __packed;Add it to the union in https://github.com/torvalds/linux/blob/master/drivers/net/can/usb/gs_usb.c#L225
Adjust the hf_size_rx if any of the device supports GS_CAN_FEATURE_HW_TIMESTAMP:
https://github.com/torvalds/linux/blob/master/drivers/net/can/usb/gs_usb.c#L1231
Add GS_CAN_MODE_HW_TIMESTAMP to flags if device supports GS_CAN_FEATURE_HW_TIMESTAMP.
https://github.com/torvalds/linux/blob/master/drivers/net/can/usb/gs_usb.c#L823
Add timestamping infrastructure like in
torvalds/linux@efd8d98
Use the usb_control_msg(GS_USB_BREQ_TIMESTAMP) to regularly read the timestamp from the device (as a replacement for mcp251xfd_get_timestamp()).
Thanks for the pointers. I'll have a look and see where I get.
@marckleinebudde I had time to look at this today and have a working fork: torvalds/linux@master...tuna-f1sh:linux:gs_usb_hwts
It's pretty much as you said. The worker grabs the reference count from the BREQ_TIMESTAMP every 3600 seconds (30 mins) - since the TIM2 is clocked at 1 MHz (rather than 48 MHz like the MCP) it would roll over after 4294 seconds (71 mins). Maybe this is overly conservative? Then each frame sets the hardware timestamp.
I've tested the function and it appears to work as intended.
Couple of questions:
- I cancel the worker in
gs_can_close. I think this is enough and will also be called on abrupt disconnect etc? (My testing seemed to indicate yes). - Do you think the timestamp functions should be broken into another file? I avoided this because it would involve adding a gs_can.h header file too and so make the patch much larger.
Let's move the discussion to the linux-can mailing list. http://vger.kernel.org/vger-lists.html#linux-can
-> https://lore.kernel.org/20220826104629.2837024-1-mkl@pengutronix.de
As far as this firmare is concerned, this can probably be closed ?
Yes closing as the firmware has everything need to support this. Will update for reference when the driver support is merged.
As promised and for reference if anyone comes across this: the gs_usb module supporting this is included in the kernel 6.1 release. Thanks to Marc for the support getting this done.
One can confirm with 6.1 that a gs_usb interface supports hardware timestamps with ethtool --show-time-stamping can0. candleLight_fw will show:
Time stamping parameters for can0:
Capabilities:
hardware-transmit
software-transmit
hardware-receive
software-receive
software-system-clock
hardware-raw-clock
PTP Hardware Clock: none
Hardware Transmit Timestamp Modes:
on
Hardware Receive Filter Modes:
all
I've used it to pin point a CAN node deviating from it's cycle threshold and then confirm that the bug was fixed. It was important to use hardware timestamping to eliminate the host OS introducing any timing delay - I found it was enough to effect my results.
For interest to demonstrate this, I just created a STM32F4 sending a CAN messages every 2 ms in the SysTick ISR. Using candleLight_fw with candump can0 -t d -H (hardware) and candump can0 -t d (software) on the same interface, the difference is clear. I did intentionally load the system during this and it's running in a VM so it's a worse case but still.
Cool, thanks for confirming and for that nice test data. How did you produce the graphs ?
The graphs are really nice!
The "hardware" RX timestamps look quite good, even if they are created by software in the µC.
For reference, current limitations:
- 1µs resolution only
- TX timestamps are created after CAN frame has been placed into TX FIFO (not after TX complete)
The graphs are produced with a Python script and plotly. Here is a folder with scatter, histogram and interactive versions if you're interested (they are big and quite slow as lots of data-points): https://www.dropbox.com/sh/sdhaht8p2cqfu7m/AAC-Y7qqVa4_G855dhWHRhssa?dl=0
Yes the hardware ones are still not perfect but good enough for most use and much better than the host timestamps for timing specific tests.
Hi @tuna-f1sh! Thanks for the awesome work here.
I have an Entree (running firmware 1.0.3 I assume) plugged into a BeagleBone Black running kernel 6.1.38, and ethtool does not show hardware timestamping support. Am I missing something? Thanks!
debian@beaglebone:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
debian@beaglebone:~$ uname -a
Linux beaglebone 6.1.38-bone22 #1buster PREEMPT Sat Jul 15 12:14:58 UTC 2023 armv7l GNU/Linux
debian@beaglebone:~$ dmesg
[...]
[ 2802.379615] usb 1-1: new full-speed USB device number 3 using musb-hdrc
[ 2802.528902] usb 1-1: New USB device found, idVendor=1d50, idProduct=606f, bcdDevice= 0.00
[ 2802.528942] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2802.528955] usb 1-1: Product: Entree gs_usb
[ 2802.528964] usb 1-1: Manufacturer: JBR Engineering
[ 2802.528974] usb 1-1: SerialNumber: 003900164858530B20353833
[ 2802.544658] gs_usb 1-1:1.0: Configuring for 1 interfaces
[ 2879.749611] c_can_platform 481d0000.can can2: setting BTR=1c02 BRPE=0000
[ 2879.755603] IPv6: ADDRCONF(NETDEV_CHANGE): can2: link becomes ready
[...]
debian@beaglebone:~$ ethtool --show-time-stamping can2
Time stamping parameters for can2:
Capabilities:
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
PTP Hardware Clock: none
Hardware Transmit Timestamp Modes: none
Hardware Receive Filter Modes: none
@lglenat I have the not very helpful reply of 'works on my machine'! I can only think that perhaps the BeagleBone kernel is not compiled with hardware timestamp support. I'm not that familiar with BeagleBone and their builds and couldn't find anything on the -bone22 tag but someone had a similar problem a while ago: https://forum.beagleboard.org/t/beaglebone-black-hardware-timestamping-with-5-10-ti-rt-kernel-using-omap-image-builder/31031
One test would be what ethtool --show-time-stamping eth0 returns. https://usermanual.wiki/Document/SetupGuide.632280828.pdf suggests that the on-board NIC supports hardware timestamps so it should return support for this. If not then the kernel is missing the support.
[ 2879.749611] c_can_platform 481d0000.can can2: setting BTR=1c02 BRPE=0000
^^^^^^^^^^^^^^ ^^^^
At least in this boot log, can2 is the C_CAN...
[ 2879.755603] IPv6: ADDRCONF(NETDEV_CHANGE): can2: link becomes ready
[...]
debian@beaglebone:~$ ethtool --show-time-stamping can2
Time stamping parameters for can2:
Capabilities:
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
PTP Hardware Clock: none
Hardware Transmit Timestamp Modes: none
Hardware Receive Filter Modes: none
... and the C_CAN doesn't support time stamping.
Sorry for the waste of time, turns out I mistakenly passed the wrong CAN interface to ethtool 🤦. I passed one of the two hardware interfaces present in the BeagleBone Black instead of the one corresponding to the Entree (or any other gs_usb-compatible device).
When I use the right one, I confirm things work as expected:
Time stamping parameters for can0:
Capabilities:
hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)
software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)
hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE)
software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)
software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: none
Hardware Transmit Timestamp Modes:
on (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
all (HWTSTAMP_FILTER_ALL)
I used the wrong one because plugging USB after the beaglebone boot enumerates the device on can2, but after a reboot it looks like gs_usb loads first and so enumerates as can0... should have checked more carefully (or find a way to map to a deterministic interface...).
Thanks all for the help.
Have a look at https://github.com/candle-usb/candleLight_fw#associating-persistent-device-names
Yep I was able to use udev successfully for this :).