Enable / Disable Routes
Closed this issue · 24 comments
The UI presents toggles for routes to be enabled or disabled but these are geyed out on all my nodes.
Hovering the mouse over them give a 'no entry symbol'
Is there something that has to be enabled to allow the controll of routes?
Hi there! There's nothing I'm aware of regarding that. The no entry symbol should only be shown while the request to enable/disable is actively being processed. Otherwise, can you give me some info?
- The version of headscale you are running
- Make sure you are using headscale-admin:latest on docker or v0.1.10b if built from source
- Is there any interesting console output? Make sure you have "console debugging" checked in the settings page
In my testing, I onboarded a device with a new route and it shows up as disabled (with 0.22.3 it tends to show up as duplicates, so I delete one of them, this is a headscale bug not headscale-admin):
The new route is 10.101.0.0/16 and I simply click on the switch to change it from red to green
Hi there! There's nothing I'm aware of regarding that. The no entry symbol should only be shown while the request to enable/disable is actively being processed. Otherwise, can you give me some info?
1. The version of headscale you are running 2. Make sure you are using headscale-admin:latest on docker or v0.1.10b if built from source 3. Is there any interesting console output? Make sure you have "console debugging" checked in the settings page
- I am running the latest headscale beta (0.23.0-Alpha5)
- Headscale-admin is latest v.0.1.10b
- I have cosole output selected --- not sure how I view the output of that.
I have attached some screen shots to ilustrate what I am seeing.
I think I have found the issue....not sure how to solve.
Cross-Origin Request Warning: The Same Origin Policy will disallow reading the remote resource at https://lheascale.domain/api/v1/node soon. (Reason: When the
Access-Control-Allow-Headersis*, theAuthorizationheader is not covered. To include theAuthorizationheader, it must be explicitly listed in CORS headerAccess-Control-Allow-Headers).
I am using Caddy as a reverse proxy and have the following rules:
@hs-options {
host hs.
method OPTIONS
}
@hs-other {
host hs.
}
handle @hs-options {
header {
Access-Control-Allow-Origin https://
Access-Control-Allow-Headers *
}
respond 204
}
handle @hs-other {
reverse_proxy http://headscale:8080 {
header_down Access-Control-Allow-Origin https://
header_down Access-Control-Allow-Headers *
}
}
Hmmm, if you are seeing the nodes within your tailnet, then it should be properly communicating with the API so I don't know why CORS would be an issue.
The API specifically pertaining to routes hasn't changed except for the nodes/machines terminology change. To me, those routes being "tunnel all" 0.0.0.0/0 routes makes me think that has something to do with it.
I think this is an advertisement issue.
See the code for whether or not that switch you're referring to is disabled:
headscale-admin/src/lib/cards/node/NodeRoute.svelte
Lines 24 to 28 in d958acd
In fact I found a bug here, but I don't believe it is what is causing your issue.
The isExpired function used to take a node/machine object, but I changed it to accept strings instead so I could check other expirations. If the page is not loading and the route is advertised, it seems like that could cause an error to be thrown.
I've updated the code and the docker image in the dev branch. Are you able to try that out?
Otherwise, the only explanation is that the route is not currently being advertised and that is why the switch is disabled. For global routes, you need to add --advertise-exit-node
I was able to replicate your issue by adding a node with --advertise-exit-node, then doing tailscale down and then tailscale up without the advertisement. When I did that, I get exactly what you see (but with route duplication bug):
So I think if you just re-join the node to the tailnet with the right advertisement, it should work as expected.
Also, you can see debugging/console output in "Inspect Element" > "Console" tab :)
Wait I am now realizing after I posted my screenshot that it clearly shows (not advertised) on each route and yours does not... ok try the dev branch and see if anything changes.
That and the machine must be online to change the state as well.
Thanks for the quick responce
I have attached screenshots of the original and dev.
Original, the issue was not just exit nodes the same behaviours was seen on any route
And now with Dev build. The node is advertising 192.168.2.0/24 but I am still unable to turn on or off.
The following message appears about 6 times in the console output:
Cross-Origin Request Warning: The Same Origin Policy will disallow reading the remote resource at https://headscale.domain.com/api/v1/user soon. (Reason: When the
Access-Control-Allow-Headersis*, theAuthorizationheader is not covered. To include theAuthorizationheader, it must be explicitly listed in CORS headerAccess-Control-Allow-Headers).
and if I enable the route via headscale cli it shows up as enabled but I can not toggle it.
Adding and removing tags works as expected.
Very intriguing. Any luck on the Inspect Element > Console output?
Going back to these lines, it would seem like one of these values results in disabled being true:
headscale-admin/src/lib/cards/node/NodeRoute.svelte
Lines 24 to 28 in d958acd
- loading is only toggled on click and has
loading = false;in a finally block. - the route does seem to be advertised or it would say (not advertised)
- the node itself is not expired
- the node is online
If all of these things are true, then I'll have to add some specific debug messages or somehow try and replicate your issue on my end. I also don't currently have a 0.23-alpha5 environment at the moment so I will work on getting one up and running.
Hey Please see screen shot for console output.
- the route does seem to be advertised or it would say (not advertised) - Yes and the route is working, as in if I enable via CLI I can reach the advertised IP address.
the node itself is not expired - not expired, I have rotated the key, Tailscale down and back up with new auth key etc. its online and can see other nodes inthe tailnet.
the node is online - yes defo online.
It's funny, I had that same message on my Mac without ever noticing a difference. but on my windows laptops, I didn't get that same error. That's what makes me think it's not related to this issue. I was able to fix it by replacing a * with Authorization,* from what I recall (I could be mistaken, it was a week ago or so).
The consle output you're seeing is actually from the browser itself, not my program. My program will use console.log(...) to write output into it from various functions, the status of various things, and any errors should be printed as well. The yellow noise is making it impossible to see anything else. If you scroll through that output, is there anything of note in there?
You should at the very least see a periodic "populating stores..." message when it fetches data to populate the svelte stores.
I will add some additional console logging output when I get a chance. I'm very curious and I definitely want to solve this for you, I'm just finding it hard to replicate. The only thing that gave me the same functionality was when I stopped advertising them.
I should also really 1) actually learn how to write unit tests someday and 2) at the very least do some testing on my macbook.
I added Authorization,* but it has not made a difference.
This is still bothering me. It's likely that you've moved on to a different headscale UI that works with your environment, but I'm just so curious as to what the root cause could be. Any chance you'd be able to ping me on discord? @GoodiesHQ on discord.
The only time I have been able to replicate exactly what you are seeing is when the node advertising that route is offline, but that doesn't seem to be the case here.
And join the headscale server! https://discord.gg/by5uFfDxUV
This is still bothering me. It's likely that you've moved on to a different headscale UI that works with your environment, but I'm just so curious as to what the root cause could be. Any chance you'd be able to ping me on discord? @GoodiesHQ on discord.
The only time I have been able to replicate exactly what you are seeing is when the node advertising that route is offline, but that doesn't seem to be the case here.
And join the headscale server! https://discord.gg/by5uFfDxUV
DM sent
Hey again! I shot you a message on discord, but in case you check GH more than discord, try out the docker image with the :dev tag if you would. I fixed a bug which certainly could be the culprit. I'm unsure as to why it hasn't been affecting many others, though!
Just pulled....exactly the same issue. It must be because I am on Headscale 23.0 Alpha 5.
It seems to have fixed all the yellow warnings that were in the previous screen shots.
I was digging around and this seemed odd.
I also have this problem with the exact same setup as OP, my workaround is setting it to:
File: headscale-admin/src/lib/cards/node/NodeRoute.svelte
// component is disabled
$: disabled =
loading ||
!route.advertised || // route is not advertised
isExpired(route.node ?? route.machine); // node is expired
// !(route.node ?? route.machine).online; // node is not online
Now it will load the switches properly. It looks like that it thinks that the nodes are offline?
Very interesting. Are you also running 0.23-alpha5? @JohanVlugt
I'll double check the reactivity on this page to make certain that the node is properly being detected as being online. Thank you for looking!
Yes, I'm also running 0.23-alpha5. One note about my work around is that it will take a second page reload (F5) before it shows a toggle able slider.
And just to be very clear, do you have the "Legacy API" value unchecked? That behavior is as if the online field no longer exists on the object, but double checking the swagger API shows that it hasn't materially changed in that version as far as I can tell.
These are my settings:
Headscale: 0.23-alpha5
Headscale-Admin: 0.1.10b with the mod.
It now works without reloading the page for some reason.
@B08Z @JohanVlugt I have figured out what the issue is. I finally migrated to alpha5 and got it working. It was a simple patch, but annoying to find.
Basically, the route endpoint that returns the route contains a field for node or machine depending on the version of headscale. That object is a direct model of the Node or Machine endpoint (again, version dependent), but in Alpha 5, it ALWAYS reports as offline, even when it is online. The node endpoint directly works just fine as expected, but the node that is received from the route only works on the 0.22.3 and simply always shows as offline for 0.23.0-alpha5. I'm not sure why, but it's clearly a bug on their side.
The workaround was to do this:
Instead of:
export let route: Route;
$: disabled =
loading ||
!route.advertised || // route is not advertised
isExpired(route.node ?? route.machine) || // node is expired
!(route.node ?? route.machine).online; // node is not online
I can just pass the node from the parent component into the child component and don't grab it from the Route object:
export let route: Route;
export let node: Node;
$: disabled =
loading ||
!route.advertised || // route is not advertised
isExpired(node.expiry || '') || // node is expired
!node.online; // node is not online
This makes the route enabling/disabling work on both pre and post 0.23.0 versions. I will push it shortly after I address some other issues and will close once it is pushed. Sorry for how long this took! 3 weeks is too long.
Thanks for all your efforts!
As soon as it's patched I'll test and report back.
Do you think it worth linking in with Headscale Dev about the bug. Seems like you have been forced to patch because of a bug in Headscale?
Thanks again. Love the UI.
Thanks for all your efforts! As soon as it's patched I'll test and report back.
Do you think it worth linking in with Headscale Dev about the bug. Seems like you have been forced to patch because of a bug in Headscale?
Thanks again. Love the UI.
I can finally close this issue with v0.1.11b
And I did add to the Known Issues section regarding the tag removal bug, but there is no issue for the routing bug at the moment. Perhaps I will open one, though. Thank you again for sticking with it so long and sorry it took so long to address!
Docker :latest works! Working on standardizing a procedure to tag the right version so as of this moment, there is no :0.1.11b tag, but there will be soon.