Broker lists services from all namespaces
Closed this issue · 8 comments
Describe the bug
Not sure if it's a bug or a feature, but we tried to build our own example broker in a second namespace running a second service broker instance.
Testing it with eden shows that each service broker lists its services and plans under v2/catalog but when requesting services (GET /v2/service_instances/…) it lists services of all other brokers too.
To Reproduce
Steps to reproduce the behavior:
- Have two brokers (A and B)
- Create service instances on broker A and B
- Request services on broker A endpoint
- You'll see instances of A and B
Expected behavior
Only services requested from the respective endpoint are shown.
Environment (please complete the following information):
- Kubernetes Version: 1.16
- Service Broker Version: 0.0.5
- Client: eden v0.7.4
There is no GET /v2/service_instances/ endpoint (i.e. https://github.com/couchbase/service-broker/blob/master/pkg/broker/broker.go#L158), you can only read a single named instance by ID. I expect the service catalog is doing the aggregation in this instance.
But assuming this is not what you mean... :D "Service instances" are registry entries, and these are secrets https://github.com/couchbase/service-broker/blob/master/pkg/registry/registry.go#L216. If you run two service brokers in the same namespace, then they will see the same service instances. If the configuration namespace is set to "" with the NAMESPACE environment variable, and the broker is given enough permissions, then it can possibly see any named instance in any namespace (is this even possible??)
The model is meant to be that the broker only manages instances in its own namespace. Do any of these scenarios sound familiar?
Hm. We actually didn't run two service brokers in the same namespace. The secrets are in two different namespaces A and B, but are listed by both brokers. Do they maybe not query them by namespace but globally select them with app=couchbase-service-broker?
It's not designed to work globally in that sense, it should only look in its own namespace, then only way to bypass that is to not specify NAMESPACE in the Deployment. I'll have a think about this and see if anything turns up.
Well, we would actually not like to bypass it. Our configmap broker just creates configmaps so far (as we are learning the ways of this project :D).
How it looks from eden
❯ eden --url https://A.broker-services.our.cluster --client us --client-secret hunter2 services
Name Service Plan Binding Broker URL
==== ======= ==== ======= ==========
couchbase-developer-couchbase-developer-private-835f96bd-d1c9-4bd3-8ec8-e867db28c154 couchbase-developer couchbase-developer-private n/a https://A.broker-services.our.cluster/
couchbase-developer-couchbase-developer-private-a32ac389-7425-40ce-8e53-a2c76adb559c couchbase-developer couchbase-developer-private n/a https://A.broker-services.our.cluster/
couchbase-developer-couchbase-developer-private-bacc3d9d-94dd-4a1e-892d-3498b7565733 couchbase-developer couchbase-developer-private n/a https://A.broker-services.our.cluster/
test-configmap-service-configmap-dc261b9a-6243-4074-b6d2-e1dcee7d4034 test-configmap-service configmap n/a https://B.broker-services.our.cluster/
test-configmap-service-configmap-55c569a6-3b1f-4fda-b498-67eec4b0eca2 test-configmap-service configmap n/a https://B.broker-services.our.cluster/
test-configmap-service-configmap-28b49df4-2d96-417a-9e0f-e07deeccb12a test-configmap-service configmap n/a https://B.broker-services.our.cluster/
❯ kubectl get pod -n couchbase
NAME READY STATUS RESTARTS AGE
couchbase-operator-864dbddd8d-z9pdn 1/1 Running 16 4h38m
couchbase-operator-couchbase-admission-controller-84846bb6cfl9g 1/1 Running 0 27h
couchbase-service-broker-886c4bf44-mt2qw 1/1 Running 0 4h40m
instance-4uzkpb40-0000 1/1 Running 0 4h28m
instance-4uzkpb40-0001 1/1 Running 0 4h28m
instance-4uzkpb40-0002 1/1 Running 0 4h27m
instance-6goyzhyu-0000 1/1 Running 0 4h1m
instance-6goyzhyu-0001 1/1 Running 0 4h1m
instance-6goyzhyu-0002 1/1 Running 0 4h1m
instance-taaaewyi-0000 1/1 Running 0 4h38m
instance-taaaewyi-0001 1/1 Running 0 4h38m
instance-taaaewyi-0002 1/1 Running 0 4h37m
❯ kubectl get pod -n configmap
NAME READY STATUS RESTARTS AGE
service-broker-7fc9d4dd6b-lfttq 1/1 Running 0 126m
I've set up two brokers in two different namespaces:
[Thu 25 Jun 09:27:02 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker svcat get brokers
NAME NAMESPACE URL STATUS
+----------------------------+-----------+----------------------------------------------+--------+
couchbase-service-broker https://couchbase-service-broker.default.svc Ready
couchbase-service-broker-2 https://couchbase-service-broker.temp.svc Ready
Each defines globally unique classes and plans:
[Thu 25 Jun 09:27:08 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker svcat get classes
NAME NAMESPACE DESCRIPTION
+-----------------------+-----------+----------------------------------------+
couchbase-developer-2 Couchbase Server developer
deployments
couchbase-developer Couchbase Server developer
deployments
[Thu 25 Jun 09:27:17 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker svcat get plans
KUBENAME:
EXTERNAL NAME:
CLASS EXTERNAL NAME:
CLASS NAME:
PLANS: [0xc0000cd860 0xc0000cdd40]
NAME NAMESPACE CLASS DESCRIPTION
+-------------------------------+-----------+-----------------------+--------------------------------+
couchbase-developer-private-2 couchbase-developer-2 Cluster accessible within the
Kubernetes cluster
couchbase-developer-private couchbase-developer Cluster accessible within the
Kubernetes cluster
Then create a service instance in each namespace:
[Thu 25 Jun 09:36:33 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker kubectl get serviceinstances
NAME CLASS PLAN STATUS AGE
test-instance ClusterServiceClass/couchbase-developer couchbase-developer-private Ready 8m
[Thu 25 Jun 09:36:39 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker kubectl get serviceinstances -n temp
NAME CLASS PLAN STATUS AGE
test-instance-2 ClusterServiceClass/couchbase-developer-2 couchbase-developer-private-2 Ready 2m
Connecting to a single service broker instance:
[Thu 25 Jun 09:36:59 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker kubectl port-forward couchbase-service-broker-6645cddc8-2zw4l 8443
I can see the local instance that it created:
[Thu 25 Jun 09:43:26 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker kubectl get serviceinstances -o yaml | grep externalID
externalID: 35cf7698-dbf9-4e9d-bc60-c3f7f3bbf53f
[Thu 25 Jun 09:43:43 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker curl -H 'X-Broker-Api-Version: 2.13' -H 'Authorization: Bearer T0p5eCr3T' -k https://localhost:8443/v2/service_instances/35cf7698-dbf9-4e9d-bc60-c3f7f3bbf53f
{"service_id":"8522e991-07bc-4225-a859-1eec1e333153","plan_id":"ec0f2c9b-0277-46d7-985f-ba1fbf3b068d","parameters":{"password":"password","size":3}}
And it can't see the instance in the other namespace:
[Thu 25 Jun 09:44:08 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker kubectl get serviceinstances -o yaml -n temp | grep externalID
externalID: c98ae84e-32a1-462c-b4ab-ce22b41ddaaa
[Thu 25 Jun 09:44:19 BST 2020] simon@symphony ~/src/github.com/couchbase/service-broker curl -H 'X-Broker-Api-Version: 2.13' -H 'Authorization: Bearer T0p5eCr3T' -k https://localhost:8443/v2/service_instances/c98ae84e-32a1-462c-b4ab-ce22b41ddaaa
{"error":"ResourceNotFound","description":"service instance does not exist"}
So as I can can see it's working as designed and I don't have an immediate recreate. If you can provide any more details on how your environment differs from my one (based off the official examples), and exhibits the problem that would be great!
I guess I can test on 1.16 rather than 1.13, but I find it highly unlikely Kubernetes would have introduced a regression.
Thanks for the thorough test… maybe we have configured some other weirdness™ - we'll try that again later in a new cleaned up cluster.
I think I see what's going on now... looking at eden you are doing a show of all services because the instance name is not specified https://github.com/starkandwayne/eden/blob/master/cmd/services.go#L21, this in turn calls this https://github.com/starkandwayne/eden/blob/master/cmd/services.go#L37, then this https://github.com/starkandwayne/eden/blob/master/cmd/opts.go#L65, then this https://github.com/starkandwayne/eden/blob/master/store/service_instances.go#L135.
So all it's doing is loading a list of stuff from a file and printing that out, nothing is filtered, and in fact all the flags you are passing in are totally ignored. Better still that has nothing to do with this at all 😸
I suspect eden works in this way so you don't forget about instances you have created, but yes, you could suggest they add filtering by service broker.
Yeah… I just noticed now that it's definitely an eden thing… because I can still see services from clusters I already destroyed. So apparently it stores the data somewhere locally.