DNS name resolving not working in kubernetes
henningWoehr opened this issue · 3 comments
henningWoehr commented
Hi, I just started using your project on kubernetes and got a problem with the newest patches
Bug description
Since the version 2.5.3, the NGINX_PROXY_PASS_URL can't be resolved anymore.
My guess is, that it is related with this #333 issue and maybe the fix of that caused this issue, but I'm not really into nginx to understand the config there.
How to Reproduce
With these manifests and the ui version set to 2.5.2, everything works fine, but any higher version doesn't work.
My container-registry manifest
apiVersion: apps/v1
kind: Deployment
metadata:
name: container-registry
namespace: default
spec:
selector:
matchLabels:
app: container-registry
template:
metadata:
labels:
app: container-registry
spec:
volumes:
- name: repos
persistentVolumeClaim:
claimName: container-registry
containers:
- name: container-registry
image: registry:2
ports:
- containerPort: 5000
env:
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
resources:
limits:
memory: "64Mi"
cpu: "200m"
requests:
memory: "15Mi"
cpu: "10m"
volumeMounts:
- name: repos
mountPath: "/var/lib/registry"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: container-registry
namespace: default
spec:
resources:
requests:
storage: 10G
accessModes:
- ReadWriteOnce
---
apiVersion: v1
kind: Service
metadata:
name: container-registry
namespace: default
spec:
selector:
app: container-registry
type: NodePort
ports:
- port: 80
targetPort: 5000
nodePort: 30500My container-registry-ui manifest
apiVersion: apps/v1
kind: Deployment
metadata:
name: container-registry-ui
namespace: default
spec:
selector:
matchLabels:
app: container-registry-ui
template:
metadata:
labels:
app: container-registry-ui
spec:
containers:
- name: container-registry-ui
image: joxit/docker-registry-ui:2.5.4
env:
- name: REGISTRY_TITLE
value: Local Registry
- name: DELETE_IMAGES
value: "true"
- name: NGINX_PROXY_PASS_URL
value: http://container-registry:80
- name: SINGLE_REGISTRY
value: "true"
resources:
limits:
memory: "32Mi"
cpu: "100m"
requests:
memory: "5Mi"
cpu: "1m"
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: container-registry-ui
namespace: default
spec:
selector:
app: container-registry-ui
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 30501If applicable, add screenshots to help explain your problem.
System information
Not neccessary in my opinion
Additional context
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/90-docker-registry-ui.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/10/10 14:34:19 [notice] 1#1: using the "epoll" event method
2023/10/10 14:34:19 [notice] 1#1: nginx/1.25.2
2023/10/10 14:34:19 [notice] 1#1: built by gcc 12.2.1 20220924 (Alpine 12.2.1_git20220924-r10)
2023/10/10 14:34:19 [notice] 1#1: OS: Linux 5.15.89-talos
2023/10/10 14:34:19 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/10/10 14:34:19 [notice] 1#1: start worker processes
2023/10/10 14:34:19 [notice] 1#1: start worker process 354
2023/10/10 14:34:19 [notice] 1#1: start worker process 355
2023/10/10 14:34:19 [notice] 1#1: start worker process 356
2023/10/10 14:34:19 [notice] 1#1: start worker process 357
127.0.0.1 - - [10/Oct/2023:14:34:54 +0000] "GET / HTTP/1.1" 200 2083 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "-"
2023/10/10 14:34:54 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:34:54 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:34:59 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:35:04 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:35:09 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:35:14 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:35:19 [error] 355#355: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
127.0.0.1 - - [10/Oct/2023:14:35:22 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "-"
2023/10/10 14:35:22 [error] 354#354: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/10 14:35:22 [error] 354#354: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
127.0.0.1 - - [10/Oct/2023:14:35:23 +0000] "GET /v2/_catalog?n=1000 HTTP/1.1" 499 0 "http://localhost:10002/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "-"
Slyke commented
Same issue for me:
2023/10/15 10:03:07 [error] 397#397: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/15 10:03:12 [error] 397#397: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/15 10:03:17 [error] 397#397: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/15 10:03:22 [error] 397#397: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/15 10:03:27 [error] 397#397: send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
2023/10/15 10:03:32 [error] 397#397: *22 docker-registry.pipelines.svc.cluster.local could not be resolved (110: Operation timed out), client: 10.7.1.229, server: localhost, request: "GET /v2/_catalog?n=1000 HTTP/1.1", host: "10.3.1.211", referrer: "http://10.3.1.211/"
10.7.1.229 - - [15/Oct/2023:10:03:32 +0000] "GET /v2/_catalog?n=1000 HTTP/1.1" 502 497 "http://10.3.1.211/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "-"
In the browser, this is returned as a 502 error.
From inside the pod:
/usr/share/nginx/html # ping google.com
PING google.com (142.251.215.238): 56 data bytes
64 bytes from 142.251.215.238: seq=0 ttl=117 time=5.823 ms
64 bytes from 142.251.215.238: seq=1 ttl=117 time=5.837 ms
^C
--- google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 5.823/5.830/5.837 ms
/usr/share/nginx/html # ping docker-registry.pipelines.svc.cluster.local.
PING docker-registry.pipelines.svc.cluster.local. (10.100.13.197): 56 data bytes
^C
--- docker-registry.pipelines.svc.cluster.local. ping statistics ---
7 packets transmitted, 0 packets received, 100% packet loss
/usr/share/nginx/html # nslookup google.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8:53
Non-authoritative answer:
Name: google.com
Address: 142.250.217.78
Non-authoritative answer:
Name: google.com
Address: 2607:f8b0:400a:804::200e
/usr/share/nginx/html # nslookup google.com
Server: 10.96.0.10
Address: 10.96.0.10:53
Non-authoritative answer:
Name: google.com
Address: 142.250.69.206
Non-authoritative answer:
Name: google.com
Address: 2607:f8b0:400a:804::200e
I'm using v2.5.4. This error does not happen when changing the version to v2.5.2
Joxit commented
Hi both of you, thank you for using my project and posting issues.
The bug is confirmed, I'm looking for the best way to support both of the use cases (standard network and kubernetes)