Exposer route gets HostAlreadyClaimed
Alveel opened this issue · 9 comments
What happened:
Upon adding the acme annotation to a route, the exposer gets created. However, the exposer route gets rejected by the default router with HostAlreadyClaimed.
Maybe duplicate of #140, but it still seems to occur even with the latest images.
What you expected to happen:
The acme controller creates appropriate routes and is able to get the certificate.
How to reproduce it (as minimally and precisely as possible):
Unsure, so far I have been able to reproduce it by adding the acme annotation to the oauth-openshift route in the openshift-authentication namespace.
Anything else we need to know?:
I have tried re-creating the router without the acme annotations with oc replace, as well as oc replace --force, and re-adding the annotations a little while later.
Environment:
- OpenShift/Kubernetes version (use
oc/kubectl version):Client Version: 4.6.16 Server Version: 4.5.27 Kubernetes Version: v1.18.3+f561b20 - Others: This is a Azure RedHat OpenShift (ARO) cluster.
I couldn't find many relevant entries in the logs, but alas:
I0216 11:47:08.835968 1 route.go:496] Started syncing Route "openshift-authentication/oauth-openshift"
I0216 11:47:08.836330 1 route.go:563] Route "openshift-authentication/oauth-openshift" needs new certificate: Route is missing CertKey
I0216 11:47:08.836553 1 route.go:607] Using ACME client with DirectoryURL "https://acme-v02.api.letsencrypt.org/directory"
I0216 11:47:09.388951 1 route.go:650] Route "openshift-authentication/oauth-openshift": Order "https://acme-v02.api.letsencrypt.org/acme/order/112525390/7925055141" is in "pending" state
I0216 11:47:09.389005 1 route.go:655] Route "openshift-authentication/oauth-openshift": Order "https://acme-v02.api.letsencrypt.org/acme/order/112525390/7925055141" contains 1 authorization(s)
I0216 11:47:09.554541 1 route.go:663] Route "openshift-authentication/oauth-openshift": order "https://acme-v02.api.letsencrypt.org/acme/order/112525390/7925055141": authz "https://acme-v02.api.letsencrypt.org/acme/authz-v3/10892179595": is in "pending" state
I0216 11:47:09.554590 1 route.go:690] route "openshift-authentication/oauth-openshift": order "https://acme-v02.api.letsencrypt.org/acme/order/112525390/7925055141": authz "https://acme-v02.api.letsencrypt.org/acme/authz-v3/10892179595": challenge "pending" is in "pending" state
I0216 11:47:09.554719 1 route.go:1008] exposer Route openshift-authentication/exposer-lkckfmiq0dn9vtl2bui4m5c8bnt65e4q98gbd9jnm6mc1cpdseh0 isn't admitted yet
I0216 11:47:09.555069 1 route.go:498] Finished syncing Route "openshift-authentication/oauth-openshift"
I think the issue with HostAlreadyClaimed stopped occurring for some reason. However the issue in the previous comment is still there (with different routes/domains). Perhaps I should create a new issue?
HostAlreadyClaimed means another namespace already has a Route with the same domain. The first namespace that claims it wins.
@tnozicka this is happening to every route i add the annotation to. i'm not sure if it's spinning up the exposer too slowly or what, but every time we add the annotation, it tries to add the exposer but it gets rejected because the main route that we added the annotation already has that route.
⚡️•100% ➜ oc version
Client Version: v4.2.0-alpha.0-657-g51011e4
Server Version: 4.6.21
Kubernetes Version: v1.19.0+2f3101c
exposer route:
spec:
host: example-nginx-default.apps.domain.com
path: /.well-known/acme-challenge/blah
to:
kind: Service
name: exposer-1n16ito7mf1f48eqoftks21tqtc7h4a5fiid238eroe5s5ogi57g
weight: 100
tls:
termination: edge
insecureEdgeTerminationPolicy: Allow
wildcardPolicy: None
status:
ingress:
- host: example-nginx-default.apps.domain.com
routerName: default
conditions:
- type: Admitted
status: 'False'
reason: HostAlreadyClaimed
message: >-
route example-nginx already exposes
example-nginx-default.apps.domain.com and is older
lastTransitionTime: '2021-05-05T20:14:47Z'
wildcardPolicy: None
routerCanonicalHostname: apps.domain.com
Can you attach the full yaml (with redacted cert if any) for both of them?
oc get route/<original> route/<exposer> -o yaml
I'm still fairly new to OpenShift (might explain my troubles) but I couldn't get the command to work so I just went to the "YAML" tab on the web interface and pulled these from there. Hopefully they're what you're looking for?
[EDIT: After posting this, I see that the command I copy/pasted out of the email that GH sent me about your update removed the <original> and <exposer> parts, explaining why the command wouldn't work. I can confirm that the YAML below is what you're requesting.]
kind: Route
apiVersion: route.openshift.io/v1
metadata:
name: example-nginx
namespace: default
selfLink: /apis/route.openshift.io/v1/namespaces/default/routes/example-nginx
uid: 7619f74b-bbac-4a69-bceb-a467aa487311
resourceVersion: '73498'
creationTimestamp: '2021-05-05T20:14:18Z'
annotations:
acme.openshift.io/status: |
provisioningStatus:
earliestAttemptAt: "2021-05-05T20:14:46.808511663Z"
orderStatus: pending
orderURI: https://acme-v02.api.letsencrypt.org/acme/order/123456789/123456789
startedAt: "2021-05-05T20:14:46.808511663Z"
kubernetes.io/tls-acme: 'true'
openshift.io/host.generated: 'true'
managedFields:
- manager: openshift-router
operation: Update
apiVersion: route.openshift.io/v1
time: '2021-05-05T20:14:19Z'
fieldsType: FieldsV1
fieldsV1:
'f:status':
'f:ingress': {}
- manager: Mozilla
operation: Update
apiVersion: route.openshift.io/v1
time: '2021-05-05T20:14:45Z'
fieldsType: FieldsV1
fieldsV1:
'f:metadata':
'f:annotations':
'f:kubernetes.io/tls-acme': {}
'f:spec':
'f:port':
.: {}
'f:targetPort': {}
'f:tls':
.: {}
'f:insecureEdgeTerminationPolicy': {}
'f:termination': {}
'f:to':
'f:kind': {}
'f:name': {}
'f:weight': {}
'f:wildcardPolicy': {}
- manager: openshift-acme-controller
operation: Update
apiVersion: route.openshift.io/v1
time: '2021-05-05T20:14:47Z'
fieldsType: FieldsV1
fieldsV1:
'f:metadata':
'f:annotations':
'f:acme.openshift.io/status': {}
spec:
host: example-nginx-default.apps.domain.com
to:
kind: Service
name: example-nginx
weight: 100
port:
targetPort: 8080
tls:
termination: passthrough
insecureEdgeTerminationPolicy: Redirect
wildcardPolicy: None
status:
ingress:
- host: example-nginx-default.apps.domain.com
routerName: default
conditions:
- type: Admitted
status: 'True'
lastTransitionTime: '2021-05-05T20:14:19Z'
wildcardPolicy: None
routerCanonicalHostname: apps.domain.comkind: Route
apiVersion: route.openshift.io/v1
metadata:
annotations:
acme.openshift.io/exposer-id: >-
example-nginx-default.apps.domain.com:/.well-known/acme-challenge/abcdefghijklmnopqrstuvwyz
acme.openshift.io/exposer-key: default/example-nginx
acme.openshift.io/status: |
provisioningStatus:
earliestAttemptAt: "0001-01-01T00:00:00Z"
orderStatus: pending
orderURI: https://acme-v02.api.letsencrypt.org/acme/order/1234567890/1234567890
startedAt: "2021-05-05T20:14:46.553448501Z"
kubernetes.io/tls-acme: 'true'
openshift.io/host.generated: 'true'
selfLink: >-
/apis/route.openshift.io/v1/namespaces/default/routes/exposer-zyxwvutsrqponmlkjihgfedcba
resourceVersion: '73495'
name: exposer-zyxwvutsrqponmlkjihgfedcba
uid: c074a3c8-fdb6-4bc1-8161-7196c3116b58
creationTimestamp: '2021-05-05T20:14:47Z'
managedFields:
- manager: Mozilla
operation: Update
apiVersion: route.openshift.io/v1
time: '2021-05-05T20:14:45Z'
fieldsType: FieldsV1
fieldsV1:
'f:spec':
'f:port':
.: {}
'f:targetPort': {}
- manager: openshift-acme-controller
operation: Update
apiVersion: route.openshift.io/v1
time: '2021-05-05T20:14:47Z'
fieldsType: FieldsV1
fieldsV1:
'f:metadata':
'f:annotations':
.: {}
'f:acme.openshift.io/exposer-id': {}
'f:acme.openshift.io/exposer-key': {}
'f:acme.openshift.io/status': {}
'f:kubernetes.io/tls-acme': {}
'f:openshift.io/host.generated': {}
'f:labels':
.: {}
'f:acme.openshift.io/exposer-uid': {}
'f:acme.openshift.io/temporary': {}
'f:ownerReferences':
.: {}
'k:{"uid":"7619f74b-bbac-4a69-bceb-a467aa487311"}':
.: {}
'f:apiVersion': {}
'f:controller': {}
'f:kind': {}
'f:name': {}
'f:uid': {}
'f:spec':
'f:host': {}
'f:path': {}
'f:tls':
.: {}
'f:insecureEdgeTerminationPolicy': {}
'f:termination': {}
'f:to':
'f:kind': {}
'f:name': {}
'f:weight': {}
'f:wildcardPolicy': {}
- manager: openshift-router
operation: Update
apiVersion: route.openshift.io/v1
time: '2021-05-05T20:14:47Z'
fieldsType: FieldsV1
fieldsV1:
'f:status':
'f:ingress': {}
namespace: default
ownerReferences:
- apiVersion: route.openshift.io/v1
kind: Route
name: example-nginx
uid: 7619f74b-bbac-4a69-bceb-a467aa487311
controller: true
labels:
acme.openshift.io/exposer-uid: 7619f74b-bbac-4a69-bceb-a467aa487311
acme.openshift.io/temporary: 'true'
spec:
host: example-nginx-default.apps.domain.com
path: /.well-known/acme-challenge/abcdefghijklmnopqrstuvwyz
to:
kind: Service
name: exposer-aqswdefrgthyjuki
weight: 100
tls:
termination: edge
insecureEdgeTerminationPolicy: Allow
wildcardPolicy: None
status:
ingress:
- host: example-nginx-default.apps.domain.com
routerName: default
conditions:
- type: Admitted
status: 'False'
reason: HostAlreadyClaimed
message: >-
route example-nginx already exposes
example-nginx-default.apps.domain.com and is older
lastTransitionTime: '2021-05-05T20:14:47Z'
wildcardPolicy: None
routerCanonicalHostname: apps.domain.comyour initial route is termination: passthrough, that's why the new one doesn't get admitted - related to #117
Awesome, thank you! Removing it seems to have worked!
My assignment ended at the company where I experienced this issue, so I'm closing it. :(