triffid/pia-wg

Device for nexthop is not up......Connection failed to stabilise

willrabbermann opened this issue · 13 comments

gentoo ~/bin # ./pia-wg.sh -f
Setting default wireguard interface name: pia
Selecting server 1 from 3 choices
Registering public key with US Silicon Valley 154.21.212.153
Updating existing interface 'pia'
RTNETLINK answers: File exists
    [Change Peer from SMmX6n3v9HsdCCNz9VxE7X9XM28NSp/uCWbsW8FczwY= to OeqCOQm7DcFq3W7GwJ+AQd5D58JGnG4HG1f4PN/Qz0s=]
    [Change pia ipaddr from 10.9.153.190 to 10.10.169.88]
Error: Device for nexthop is not up.
Error: Device for nexthop is not up.
PIA Wireguard 'pia' configured successfully
Waiting for connection to stabilise......Connection failed to stabilise, try again

I do not have a pia route

gentoo ~/bin # ip route
default via 192.168.1.1 dev eth0 metric 5
10.0.0.0/24 dev wg0 proto kernel scope link src 10.0.0.1
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.2

ip link set pia up
Gave me route,
default dev pia scope link
Set pia to a lower metric than my eth0 and wg0 devs. Still ipleak.net shows my real ip weird.

gentoo ~/bin # ./pia-speedtest.sh
Setting default wireguard interface name: pia
Ping: ~112.404027123ms.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 11 71.1M   11 8494k    0     0   821k      0  0:01:28  0:00:10  0:01:18 1209k
curl: (28) Operation timed out after 10337 milliseconds with 8698539 out of 74579476 bytes received

Can run a download test but can't upload.

Error: Device for nexthop is not up.

Wow, never seen that before.. wireguard interface is 'up' straight after its created here.

Set pia to a lower metric than my eth0 and wg0 devs. Still ipleak.net shows my real ip

What does ip route get 4.2.2.1 say after doing that?

Can run a download test

Doesn't mean much if it's not testing through pia ;)

gentoo ~/bin # ip route get 4.2.2.1
4.2.2.1 dev pia src 10.17.225.12 uid 0
    cache
gentoo ~/bin # route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         0.0.0.0         0.0.0.0         U     0      0        0 pia
0.0.0.0         192.168.1.1     0.0.0.0         UG    10     0        0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 wg0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

net.pia is * status: started

I have wg0 set for my local area network connections. Wonder why its not applying my metric to wg0 or pia. I feel its close to working. :) Im using Netifrc to manage my interfaces

# ip route get 4.2.2.1
4.2.2.1 dev pia

ipleak et al shouldn't see your ISP connection then…

Wonder why its not applying my metric to wg0 or pia.

old school route doesn't do metrics afaik, the more modern ip route (as used in your earlier comments) is the tool to check :p

Im using Netifrc to manage my interfaces

Is that the openrc /etc/init.d/net stuff? If not, I have no idea what that is.

ipleak et al shouldn't see your ISP connection then…

Unfortunately I'm still getting detection's from my home address, not pia's servers.

Is that the openrc /etc/init.d/net stuff? If not, I have no idea what that is.

Yep thats it! I put the net.pia script into /etc/init.d/* and started it.

Unfortunately I'm still getting detection's from my home address, not pia's servers.

Something weird is going on with your routing then - if ip route get … for non-local IPs is saying it's gonna use pia, then all your outgoing internet requests should also use pia

Got some ip rules or iptables stuff or something that might be interfering?

As you can tell Im a networking newb.

I have not set any ip rules, though someone on the #wireguard irc told me I should look into adding a new rule. My iptables is unchanged as well.

I believe what I need is to add an ip rule to the RPDB; although I just need to learn how to use it. :)

I have not set any ip rules

pia-wg adds all the rules it needs, namely

10:     from all fwmark 0xca6c lookup 51820        // forces outgoing wireguard packets to hardware link table to avoid infinite recursion
10:     from all to «pia endpoint IP» lookup 51820 // forces HTTPS setup calls to PIA's API to use hardware link instead of wireguard

From these code lines:

[ "$EUID" -eq 0 ] && [ -z "$OPT_CONFIGONLY" ] && ip rule add to "$WG_HOST" lookup $HARDWARE_ROUTE_TABLE pref 10

ip rule add fwmark 51820 lookup "$HARDWARE_ROUTE_TABLE" pref 10

ip rule add fwmark 51820 lookup "$HARDWARE_ROUTE_TABLE" pref 10

And also creates the appropriate hardware link routing table at

pia-wg/pia-wg.sh

Lines 243 to 264 in 91bda8c

if ! ip route show table "$HARDWARE_ROUTE_TABLE" 2>/dev/null | grep -q .
then
ROUTES_ADD=$(
for IF in $(ip link show | grep -B1 'link/ether' | grep '^[0-9]' | cut -d: -f2)
do
ip route show | grep "dev $IF" | sed -e 's/linkdown//' | sed -e "s/^/ip route add table $HARDWARE_ROUTE_TABLE /"
done
)
if [ "$EUID" -eq 0 ]
then
sh <<< "$ROUTES_ADD"
else
echo "Build a routing table with only hardware links to stop wireguard packets going back through the VPN:"
echo sudo sh '<<<' "$ROUTES_ADD"
sudo sh <<< "$ROUTES_ADD"
fi
echo "Table $HARDWARE_ROUTE_TABLE (hardware network links) now contains:"
ip route show table "$HARDWARE_ROUTE_TABLE" | sed -e "s/^/${TAB}/"
echo
echo "${BOLD}*** PLEASE NOTE: if this table isn't updated by your network post-connect hooks, your connection cannot remain up if your network links change${NORMAL}"
echo "Managing such hooks is beyond the scope of this script"
fi
if you haven't already got one

I believe what I need is to add an ip rule to the RPDB

What's RPDB? Google says some sort of remote python debugger…?

Routing Policy Database (RPDB), and it seems your script creates a new rule every time its started or reconnects:

will@gentoo ~ λ sudo ip rule show
0:	from all lookup local
10:	from all to 156.146.60.136 lookup 51820
10:	from all fwmark 0xca6c lookup 51820
32766:	from all lookup main
32767:	from all lookup default

If this is the case, I am even more confused because it should be working.
Still getting ip leaks from my ISP. Really not sure where to go from here :(

Still getting ip leaks from my ISP

Is it a DNS leak?

I don't think my script even attempts to handle DNS setup, but you could pull them from $CONFIGDIR/remote.info in a post-connect hook and do something with them if you're not running a local resolver or using public DNS eg 1.1.1.1.

Really not sure where to go from here

When I start running out of ideas for weird network stuff, usually that means it's time to dump everything from ip route show table all and ip rule (and maybe iptables-save) in a text file or something, then fire up wireshark and see where packets are actually being sent and what their source addresses are while tracing through the various firewall rules to see how any weird ones could end up there.

ipleak.net says all clean when I try it fwiw.

Got it working? Care to post your solution in case someone else has a similar issue in the future?

I'm actually having a similar issue with the same error ... @willrabbermann, can you post what you did to get it working?