Wireguard non-functional
KittyKatt opened this issue · 11 comments
Hey there, I've had edge3 installed for some time now on my UDM Pro. Over the last weeks, my UDM auto updated to 1.11.4 firmware (which I thought I disabled? too late now lol). The first thing I went about doing was figuring out why my multipath routing wasn't working anymore and come to find that the routes are being rejected, which led me to believe I was back on stock kernel. Sure enough. So, I rebooted into edge3 with the newest udm-kernel-tools package and I have multipath routing back! Yay!
However, since this reboot wireguard has been totally non-functional. I get this when attempting to bring up the wg0 interface:
Warning: `/mnt/data/wireguard/etc/wireguard/wg0.conf' is world accessible
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.10.10.2 dev wg0
[#] ip link set mtu 1420 up dev wg0
Error: mtu less than device minimum.
[#] ip link delete dev wg0
After tons of trying, I rebooted back into the stock kernel and tried to use the wireguard-kmod utility again. It loaded the kernel modules in there and functions great, but I then have no multipathing.
This worked right up until the auto update to v1.11.4. Is there any reason this could now be broken for me?
For reference, the stock kernel is: 4.19.152-al-linux-v10.2.0-v1.11.4.3940-e66d85f
Additionally, I actually had to edit the 10-setup_wireguard on_boot configuration to include the following, because it wasn't finding any kernel modules for wireguard whatsoever before adding this:
elif [ -e /lib/modules/$ver/kernel/net/wireguard/wireguard.ko ]; then
modprobe wireguard
This script worked before the update to v1.11.4, so I'm unsure HOW it was functioning before this.
Did you check whether the module is actually loaded? (e.g., lsmod)
lsmod shows that the kernel module is loaded.
Module Size Used by
wireguard 126976 0
ip6_udp_tunnel 16384 1 wireguard
udp_tunnel 16384 1 wireguard
xt_conntrack 16384 8
nf_nat_tftp 16384 0
nf_conntrack_tftp 16384 1 nf_nat_tftp
nf_nat_pptp 16384 0
nf_conntrack_pptp 16384 1 nf_nat_pptp
nf_nat_h323 20480 0
nf_conntrack_h323 53248 1 nf_nat_h323
nf_nat_proto_gre 16384 1 nf_nat_pptp
nf_conntrack_proto_gre 16384 1 nf_conntrack_pptp
nf_nat_ftp 16384 0
nf_conntrack_ftp 16384 1 nf_nat_ftp
nf_log_ipv4 16384 2
nf_log_common 16384 1 nf_log_ipv4
xt_LOG 16384 2
nf_app 16384 0
t_miner 49152 1 nf_app
tdts 438272 1 t_miner
xt_dpi 16384 0
ifb 16384 0
sch_htb 24576 0
ppp_generic 36864 0
slhc 20480 1 ppp_generic
lzo 16384 4
lzo_compress 16384 1 lzo
zram 24576 1
gpiodev 16384 0
ubnthal 1253376 0
ubnt_common 102400 1 ubnthal
Could you check what the minimum MTU is for wg0? Perhaps that might give some hint.
I did:
ip link add wg0 type wireguard
ip -4 address add 10.10.10.2 dev wg0
ip addr:
34: wg0: <PROMISC> mtu 2147483552 qdisc noop state DOWN group default qlen 1000
link/infiniband
inet 10.10.10.2/32 scope global wg0
valid_lft forever preferred_lft forever
That's a pretty wild MTU.
Some additional information about the link:
36: wg0: <PROMISC> mtu 2147483552 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/infiniband promiscuity 0
wireguard addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
Very interesting 🤔 What also surprises me is that it reports a link over Infiniband.
Since it worked with the previous firmware, I suspect that there are some incompatibilities between Ubiquiti's proprietary kernel modules and the edge3 kernel.
I am working on developing shims for these module, so that this problem should not happen as often. I have sent Ubiquiti a request for the latest kernel sources so that I can update the custom kernel.
In the meantime, you could perhaps downgrade your firmware installation back to v1.11.0 if you need this to work.
I will probably end up downgrading for now. Could you keep this issue open and tag me when you've got the latest source figured out and a new custom kernel built?
Hi @KittyKatt, I managed to reproduce your issue and verify it with the new kernel sources. It appears the kernel itself is not the problem (there were little changes for v1.11.4). Instead, what happens is that modprobe selects the incorrect wireguard.ko file (which is now included in the stock firmware).
You can verify this yourself as follows:
insmod /lib/modules/4.19.152-edge3/kernel/net/wireguard/wireguard.koThis should load the right kernel module. When creating a new interface now, Linux should report the expected values.
I'll fix the issue in udm-kernel-tools so that this will not happen anymore.
Fixed with fabianishere/udm-kernel-tools#65
@fabianishere Just saw this. I'm going to hopefully be trying this out and validating this weekend. Thank you!