SPN outage? failed to ping home hub: timed out
northys opened this issue · 10 comments
What happened:
All I remember is that I added new exclude rule for SPN and few minutes later (around 19:40 CEST) SPN including my WG VPN started timeouting. I wondered if it has something in common with safing/portmaster#566 as it behaved exactly as I described there.
I think that my home network was fine. I could browse the internet on my phone while this happened on my laptop.
What did you expect to happen?:
SPN do not timeout / report it is not ready for tunneling.
How did you reproduce it?:
:/
Debug Information:
https://support.safing.io/privatebin/?48b5e49f042c52a4#HYe7DLu2YdiC4snb172VPozdknxu8YdM4FSgCchMybxk
I think that this is just another issue for safing/portmaster#566 because restart fixed it. I barely could use my LAN, VPN nor SPN. It was working for like 5 seconds/1 minute. I have no idea what is happening with my network on my laptop...
Hm. It seems that the health check triggered a couple reconnects. Were you downloading stuff or trying to?
220402 19:37:51.200 ain/client:426 ▶ WARN 211 spn/captain: failed to ping home hub: timed out
220402 19:40:07.339 ain/client:426 ▶ WARN 732 spn/captain: failed to ping home hub: timed out
220402 19:40:23.524 ain/client:426 ▶ WARN 939 spn/captain: failed to ping home hub: timed out
220402 19:40:39.009 ain/client:426 ▶ WARN 131 spn/captain: failed to ping home hub: timed out
220402 19:40:54.357 ain/client:426 ▶ WARN 312 spn/captain: failed to ping home hub: timed out
220402 19:42:04.339 CURRENT TIME
I forgot the mention: There were no irregularities on the SPN nodes during the time frame in question. Everything seems normal.
Were you downloading stuff or trying to?
No, I just browsed internet, no video streams etc. But I streamed video over LAN from my home server.
I just cant believe its caused by something in my LAN. That would cause troubles also on my phone (I was able to measure 220/180Mbps in librespeed hosted on mentioned server) and internet went fine as well.
I also tried to stop portmaster but it did not help. I had to reboot my laptop and it got fixed.
When I checked node exporter graphs from my laptop there is nothing. No packet drops, no errors, just the speed is slow...
There must be something wrong but as I mentioned in the linked issue it does NOT have to be caused by portmaster.
Internet got slow (but still worked), no huge file downloading. Turning it off made the internet work again. Connecting again fixed also the SPN.
https://support.safing.io/privatebin/?bc96656d64c931d2#EUdoVofC5JqqwTeRsVUyqUaB5ne3sM2jgKmpN9HZ9iaT
220405 02:19:01.899 /operation:191 ▶ WARN 505 spn/terminal: operation connect e681f6#8#24#0>608 failed: [ext] connection error
220405 02:33:27.206 /operation:191 ▶ WARN 855 spn/terminal: operation connect e681f6#8#24#0>1992 failed: [ext] connection error
220405 02:33:30.525 /operation:191 ▶ WARN 876 spn/terminal: operation connect e681f6#8#24#0>1888 failed: [ext] connection error
220405 03:15:14.557 /operation:191 ▶ WARN 762 spn/terminal: operation connect e681f6#8#24#0>2848 failed: [ext] connection error
220405 04:21:16.143 dbus_linux:065 ▶ WARN 637 failed to get nameserver: failed to access /:org.freedesktop.NetworkManager.Connection.Active.Ip4Config: Object does not exist at path “/”
220405 04:21:16.143 dbus_linux:072 ▶ WARN 639 failed to get nameserver: failed to access /:org.freedesktop.NetworkManager.Connection.Active.Ip6Config: Object does not exist at path “/”
220405 14:40:01.127 ain/client:426 ▶ WARN 647 spn/captain: failed to ping home hub: timed out
220405 14:40:01.178 v/location:288 ▶ WARN 650 netenv: failed to get IPv4 device location from traceroute: failed to send icmp packet: write ip4 0.0.0.0->1.1.1.1: sendto: network is unreachable
220405 14:40:01.180 ain/client:323 ▶ ERRO 652 failed to establish connection to home hub: failed to connect to a new home hub - tried 10 hubs: failed to launch ship: failed to connect to <Hub calla UVe7-iGR9> using tcp:17 (144.217.85.240): dial tcp 144.217.85.240:17: connect: network is unreachable
220405 14:43:38.267 /operation:191 ▶ WARN 330 spn/terminal: operation connect 8d36ed#8#16#0>560 failed: [ext] connection error
220405 14:44:02.390 CURRENT TIME
Hm. I think this might be something with the flow control, which I am currently looking into.
Thanks for the updates.
Do you think that would also affect wireguard connections or LAN connections?
LAN connections should not be affected. Wireguard would only be affected if it goes through the SPN.
Were you downloading stuff or trying to?
No, I just browsed internet, no video streams etc. But I streamed video over LAN from my home server.
I just cant believe its caused by something in my LAN. That would cause troubles also on my phone (I was able to measure 220/180Mbps in librespeed hosted on mentioned server) and internet went fine as well.
I also tried to stop portmaster but it did not help. I had to reboot my laptop and it got fixed.
When I checked node exporter graphs from my laptop there is nothing. No packet drops, no errors, just the speed is slow...
There must be something wrong but as I mentioned in the linked issue it does NOT have to be caused by portmaster.
Could also have been a Windows/Linux problem.
Flow control was revamped early 2023 - while timeouts still happen, they are a lot less frequent.
If this still happens, please open a new issue.