Configuration with self-hosted Unifi application application
Closed this issue · 11 comments
Bug Description
I am trying to configure external-dns-unifi-webhook for my network, which consists of:
- an USG-Pro 4 router connected to
- the Unifi Network Server running in Kubernetes via jacobalberty/unifi-docker.
The first step in the README says to:
Create a local user with a password in your UniFi OS, this user only needs read/write access to the UniFi Network appliance.
I am not sure what this means in my case. The "local user [...] in your UniFi OS" refers to a user on my USG-Pro 4 device (e.g. over SSH), which would not have access to the "Unifi Network appliance."
I tried two different configurations for the UNIFI_HOST, username, and password options:
UNIFI_HOSTset to the IP address of my USG-Pro 4 router, withusernameandpasswordset to the SSH login credentials -- configured in the "Device SSH Authentication" section of my Unifi Network Server.UNIFI_HOSTset to the hostname of my Unifi Network Server, with theusernameandpasswordI use to login to its web interface.
Neither worked. The USG-Pro 4 configuration succeeds at logging in, but repeatedly fails to query /proxy/network/v2/api/site/default/static-dns with an HTTP 404 error. The Unifi Network Server configuration repeatedly fails to query /api/auth/login with an HTTP 401 error.
ExternalDNS Configuration
Helm values
For External DNS Helm chart version 1.14.5, installed in the network namespace:
# Configures https://github.com/kashalls/external-dns-unifi-webhook
fullnameOverride: external-dns-unifi
logLevel: debug
provider:
name: webhook
webhook:
image:
repository: ghcr.io/kashalls/external-dns-unifi-webhook
tag: v0.2.1
env:
- name: UNIFI_HOST
# For USG-Pro 4
#value: https://192.168.1.1
# For Unifi Network Server:
value: https://unifi-app.network.svc.cluster.local:8443
- name: UNIFI_USER
valueFrom:
secretKeyRef:
name: external-dns-unifi
key: username
- name: UNIFI_PASS
valueFrom:
secretKeyRef:
name: external-dns-unifi
key: password
- name: LOG_LEVEL
value: debug
livenessProbe:
httpGet:
path: /healthz
port: http-wh-metrics
initialDelaySeconds: 10
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /readyz
port: http-wh-metrics
initialDelaySeconds: 10
timeoutSeconds: 5
extraArgs:
- --ignore-ingress-tls-spec
policy: sync
sources:
- ingress
- service
txtOwnerId: default
txtPrefix: k8s.
domainFilters:
- mydomain.com # TODO parameterizeSecret
apiVersion: v1
kind: Secret
metadata:
name: external-dns-unifi
namespace: network
labels:
argocd.argoproj.io/instance: prod-external-dns
type: Opaque
data:
username: mypassword
password: myusernameExternalDNS Logs
With UNIFI_HOST set to the USG-PRO-4 hostname:
time="2024-10-13T14:36:28Z" level=info msg="config: {APIServerURL: KubeConfig: RequestTimeout:30s DefaultTargets:[] GlooNamespaces:[gl
time="2024-10-13T14:36:28Z" level=info msg="Instantiating new Kubernetes client"
time="2024-10-13T14:36:28Z" level=debug msg="apiServerURL: "
time="2024-10-13T14:36:28Z" level=debug msg="kubeConfig: "
time="2024-10-13T14:36:28Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2024-10-13T14:36:28Z" level=info msg="Created Kubernetes client https://10.96.0.1:443"
time="2024-10-13T14:36:28Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T14:36:29Z" level=debug msg="Failed to get records with code 500"
time="2024-10-13T14:36:29Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
time="2024-10-13T14:37:29Z" level=debug msg="Failed to get records with code 500"
time="2024-10-13T14:37:29Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
With UNIFI_HOST set to the Unifi network application hostname:
time="2024-10-13T15:03:59Z" level=info msg="config: {APIServerURL: KubeConfig: RequestTimeout:30s DefaultTargets:[] GlooNamespaces:[gl
time="2024-10-13T15:03:59Z" level=info msg="Instantiating new Kubernetes client"
time="2024-10-13T15:03:59Z" level=debug msg="apiServerURL: "
time="2024-10-13T15:03:59Z" level=debug msg="kubeConfig: "
time="2024-10-13T15:03:59Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2024-10-13T15:03:59Z" level=info msg="Created Kubernetes client https://10.96.0.1:443"
time="2024-10-13T15:03:59Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T15:04:00Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T15:04:01Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T15:04:02Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T15:04:03Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T15:04:05Z" level=debug msg="Failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
time="2024-10-13T15:04:05Z" level=fatal msg="failed to connect to webhook: Get \"http://localhost:8888\": dial tcp [::1]:8888: connect
Stream closed EOF for network/external-dns-unifi-59cc99976f-rfg5d (external-dns)
Webhook Logs
With UNIFI_HOST set to the USG-PRO-4 hostname:
external-dns-provider-unifi
version: v0.2.1 (44717b332013999204c93252095972a98188d021)
{"level":"info","ts":1728830188.7894733,"caller":"log/log.go:49","msg":"creating unifi provider with no kind of domain filters"}
{"level":"debug","ts":1728830188.7898133,"caller":"log/log.go:53","msg":"making POST request to https://192.168.1.1/api/auth/login"}
{"level":"debug","ts":1728830188.789953,"caller":"log/log.go:53","msg":"Requesting https://192.168.1.1/api/auth/login cookies: 0"}
{"level":"debug","ts":1728830189.12711,"caller":"log/log.go:53","msg":"response code from POST request to https://192.168.1.1/api/auth
/login: 200"}
{"level":"info","ts":1728830189.1275892,"caller":"log/log.go:49","msg":"starting webhook server","address":"localhost:8888"}
{"level":"info","ts":1728830189.1277475,"caller":"log/log.go:49","msg":"starting health server","address":"0.0.0.0:8080"}
{"level":"debug","ts":1728830189.3808708,"caller":"webhook/webhook.go:100","msg":"requesting records","req_method":"GET","req_path":"/
records"}
{"level":"debug","ts":1728830189.3810122,"caller":"log/log.go:53","msg":"making GET request to https://192.168.1.1/proxy/network/v2/ap
i/site/default/static-dns/"}
{"level":"debug","ts":1728830189.3810823,"caller":"log/log.go:53","msg":"Requesting https://192.168.1.1/proxy/network/v2/api/site/defa
ult/static-dns/ cookies: 1"}
{"level":"debug","ts":1728830189.5920637,"caller":"log/log.go:53","msg":"response code from GET request to https://192.168.1.1/proxy/n
etwork/v2/api/site/default/static-dns/: 404"}
{"level":"error","ts":1728830189.5921562,"caller":"webhook/webhook.go:104","msg":"error getting records","req_method":"GET","req_path"
:"/records","error":"GET request to https://192.168.1.1/proxy/network/v2/api/site/default/static-dns/ was not successful: 404","stackt
race":"github.com/kashalls/external-dns-provider-unifi/pkg/webhook.(*Webhook).Records\n\t/build/pkg/webhook/webhook.go:104\nnet/http.H
andlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2171\ngithub.com/go-chi/chi/v5.(*Mux).routeHTTP\n\t/go/pkg/mod/github.com
/go-chi/chi/v5@v5.1.0/mux.go:459\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2171\ngithub.com/go-chi/chi/v
5.(*Mux).ServeHTTP\n\t/go/pkg/mod/github.com/go-chi/chi/v5@v5.1.0/mux.go:90\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net
/http/server.go:3142\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2044"}
With UNIFI_HOST set to the Unifi network application hostname:
local:8443/api/auth/login"}
{"level":"debug","ts":1728831823.7233777,"caller":"log/log.go:53","msg":"Requesting https://unifi-app.network.svc.cluster.local:8443/a
pi/auth/login cookies: 0"}
{"level":"debug","ts":1728831823.7344944,"caller":"log/log.go:53","msg":"response code from POST request to https://unifi-app.network.
svc.cluster.local:8443/api/auth/login: 401"}
{"level":"debug","ts":1728831823.734542,"caller":"log/log.go:53","msg":"Received 401 Unauthorized, re-login required"}
{"level":"debug","ts":1728831823.734568,"caller":"log/log.go:53","msg":"making POST request to https://unifi-app.network.svc.cluster.l
ocal:8443/api/auth/login"}
{"level":"debug","ts":1728831823.7346292,"caller":"log/log.go:53","msg":"Requesting https://unifi-app.network.svc.cluster.local:8443/a
pi/auth/login cookies: 0"}
{"level":"debug","ts":1728831823.7456665,"caller":"log/log.go:53","msg":"response code from POST request to https://unifi-app.network.
svc.cluster.local:8443/api/auth/login: 401"}
{"level":"debug","ts":1728831823.745711,"caller":"log/log.go:53","msg":"Received 401 Unauthorized, re-login required"}
{"level":"debug","ts":1728831823.745742,"caller":"log/log.go:53","msg":"making POST request to https://unifi-app.network.svc.cluster.l
ocal:8443/api/auth/login"}
{"level":"debug","ts":1728831823.7457945,"caller":"log/log.go:53","msg":"Requesting https://unifi-app.network.svc.cluster.local:8443/a
pi/auth/login cookies: 0"}
UniFiOS Version
4.4.57.5578372
UniFi Network Version
8.3.32
ExternalDNS Version
v0.14.2
Search
- I have searched the open and closed issues for duplicates
Code of Conduct
- I agree to follow this project's Code of Conduct
Hey @mkoval, unfortunately it does not look like the dockerized version of the unifi controller supports DNS functionality.
"UniFi OS" refers to the special operating system available with UniFi's cloud keys/gateways listed here. The USG-Pro-4 lacks this functionality unfortunately.
The USG-Pro-4 was released in 2014 and some of its predecessor's have already been marked as EOL, I would recommend upgrading if possible.
Alternatively, external-dns supports a "wide range" of dns services. You can get around using an additional dns server on the network like pihole, dnsmasq or powerdns, forwarding your dns from the USG-Pro-4 to the extra dns server.
I see. You appear to be correct, given that the device greets me over SSH with the following welcome message: "Welcome to EdgeOS on UniFi Security Gateway!" I was mislead because the device is branded as the Unifi Security Gateway, rather than the Edge Security Gateway, but oh well -- it is a decade old now.
I notice that the CloudKey+ is listed on that page, which should theoretically be able to manage my existing USG Pro 4. I am not sure if that configuration would work. It might depend on (a) which device the DNS server runs on and (b) whether the DNS feature is enabled when using an older router. Do you know if this configuration would work?
If I were to upgrade to a newer router, I would probably opt for the Dream Machine Pro (UDM-Pro). Do you know if the UDM-Pro runs UnifiOS? I assume that it would, but it is listed under UniFi Cloud Gateways rather than the CloudKeys & Gateways page you linked it. It is not clear to me why these are different.
Thanks for the quick response and your help.
I see notice that the CloudKey+ is listed on that page, which should theoretically be able to manage my existing USG Pro 4. I am not sure if that configuration would work. It might depend on (a) which device the DNS server runs on and (b) whether the DNS feature is enabled when using an older router. Do you know if this configuration would work?
I believe it has to be a feature set that is enabled on the device itself. Switching to the CloudKey+ would provide the API, but I fear that the feature set is not implemented on the USG Pro 4. You can check to see by going to Network > Settings > Routing > DNS and creating a record to test.
If I were to upgrade to a newer router, I would probably opt for the Dream Machine Pro (UDM-Pro). Do you know if the UDM-Pro runs UnifiOS? I assume that it would, but it is listed under UniFi Cloud Gateways rather than the CloudKeys & Gateways page you linked it. It is not clear to me why these are different.
It would. I have tested this with the UDM Pro Max + UXG-MAX (requires separate controller). Other user's have reported similar success with the devices listed on the CloudKeys & Gateways page.
Feel free to inquire in the Home Operations discord or here.
I cant get it working on https://store.ui.com/us/en/category/all-cloud-keys-gateways/products/uxg-max, no option to create local user.
I cant get it working on https://store.ui.com/us/en/category/all-cloud-keys-gateways/products/uxg-max, no option to create local user.
You'll need to use a controller device with it in order for it to work. The UXG Max does not have a Unifi Controller built into it. It's only the router.
On the network console menu go to settings, users and create a user. Select restrict to local access only.
I cant get it working on https://store.ui.com/us/en/category/all-cloud-keys-gateways/products/uxg-max, no option to create local user.
You'll need to use a controller device with it in order for it to work. The UXG Max does not have a Unifi Controller built into it. It's only the router.
On the network console menu go to settings, users and create a user. Select restrict to local access only.
Thanks for quick response, is it possible to directly interact with unifi network server and add DNS entries on that?
Unfortunately this project can only exist due to Ubiquiti adding API support for managing DNS records which requires a device that can act as a UniFi controller and has the full UniFi OS.
@onedr0p I understand but this is from my unifi network server running on ip 192.168.1.5. Here are screenshots of postman request sent to my unifi network controller running on rpi5.
One thing to note, I only have unifi gateway max, I dont have any other unifi device. I dont know how this behaviour will change if I get more unifi devices.
Query all DNS records using get https://192.168.1.5:8443/v2/api/site/default/static-dns
Now if we make a post request to https://192.168.1.5:8443/v2/api/site/default/static-dns
To remove entry just send delete verb to same url https://192.168.1.5:8443/v2/api/site/default/static-dns
If incase this is something supported, we would need an env variable to toggle between if its unifi controller or network server.
Not a GO expert but got it working using Unifi Network server when creating a simple ingress:
Delete record also works
