Proxy environment variable doesn't seem to work
Closed this issue · 10 comments
I tried launching the container with the optional "proxy" environment variable being set to true but it doesn't seem to proxy the requests.
The environment in which the container is being launched in has access to the registry. The environment from which i'm accessing the UI doesn't have access to the registry. This is why i'm trying to use the proxy variable.
The proxy variable works in other landoop tools such as the kafka-topics-ui but it doesn't seem to be working with the schema registry ui.
The only environment variables i'm passing in are SCHEMAREGISTRY_URL and PROXY. I'm setting the proxy value to 'true' and the shemaregistry_url to my schema registry url.
When loading the schema registry UI, i can see the requests for subjects and config at my schema registry URL in firebug. Those request are timing out because i need it to be proxied via the schema registry ui.
What am I missing?
Did you include a scheme for your schema registry? I.e you should use http://address instead of just address.
Let's debug a bit. Following the instructions below, you (a) will find the ID of the Schema Registry UI container, (b) enter into it, (c) check you have set the correct variables, (d) install curl, (e) check that schema registry is accessible.
$ docker ps
$ docker exec -it [container_id] sh
/ # printenv
/ # apk add --no-cache curl
/ # curl "$SCHEMAREGISTRY_URL"
If all went well, the last command should return {}. Also make sure $SCHEMAREGISTRY_URL starts with the http scheme.
Yes, i'm passing in the full URL. I have the schema registry UI working in our dev environment because we allow direct access to the schema registry but we don't allow it in prod. Kafka topics ui works in prod because the proxy option seems to be working as expected but the schema registry proxy doesn't seem to proxy all requests through the container. Its trying to access the schema registry directly to pull the config and the subjects.
Here are some of the environment variables in the container and the curl output showing that the container itself is able to connect to the registry.
SCHEMA_REGISTRY_UI_PORT_8000_TCP=tcp://172.30.137.48:8000
SCHEMAREGISTRY_URL=https://hostname-removed-but-is-valid
SCHEMA_REGISTRY_UI_SERVICE_PORT=8000
SCHEMA_REGISTRY_UI_PORT=tcp://172.30.137.48:8000
HOSTNAME=schema-registry-ui-4-fzf0s
SCHEMA_REGISTRY_UI_PORT_8000_TCP_ADDR=172.30.137.48
SCHEMA_REGISTRY_UI_PORT_8000_TCP_PORT=8000
SCHEMA_REGISTRY_UI_SERVICE_PORT_SCHEMA_REGISTRY_UI=8000
SCHEMA_REGISTRY_UI_PORT_8000_TCP_PROTO=tcp
SCHEMA_REGISTRY_UI_SERVICE_HOST=172.30.137.48
curl https://hostname-removed-but-is-valid
The container is running on kubernetes.
# printenv | grep -i proxy
PROXY=true
Maybe your k8s doesn't serve Schema Registry UI from the address root?
The PROXY option can work only if you serve Schema Registry UI from /.
For example it will work if kubernetes serves it from http://my.address.url/ but not if it serves it from http://my.address.url/schema-registry-ui.
The reason for this is that in order to proxy requests to Schema Registry we have to remove the non schema registry specific part of the URL (/api/schema-registry). If you don't serve under / we don't have a way to know what you added.
Another possibility is that you run the Schema Registry with a self issued certificate. In that case set the env variable INSECURE_PROXY=1.
Its served under /.
Firebug shows its trying to access the schema registry directly and not on /api/schema-registry of the schema registry ui.
Its setup the same exact way as the kafka topics ui and the proxying there works.
The certs are not self signed. But regardless if they were or not, the URL shown in firebug in my browser when accessing the UI says its trying to load content directly from the schema registry url which is incorrect. That means the proxy option isn't working.
Here is what i see in my browser. the schema registry is at https://stage-confluent-registry.... and the UI is at https://schema-registry-ui-stage-ds....
Its trying to access the config from the actual schema registry url which will never load due to network limitations in place. The schema registry UI container, however, can access the schema registry.
It is very funny. Could you check that caddy (the web server inside the container) is set correctly?
docker exec [container_id] cat /caddy/Caddyfile
You should see something like:
0.0.0.0:8000
tls off
root /schema-registry-ui
log /access.log
proxy /api/schema-registry https://stage-confluent-registry {
without /api/schema-registry
}
I can't figure out why your browser tries to connect directly to the schema registry. Maybe it is a cache issue? If you are on Chrome, try to open the developer console (CTRL+SHIFT+i) and then press refresh. This will clear the cache.
I just rebuilt the container and it seems to be working now. I was using a container that was built about 2 months ago. Now its accessing the correct URIs via /api/ on the ui.
thanks for helping me troubleshoot this!
Glad we tackled it!
Btw, bit of a shameless plug, but in the next few days we will release our enterprise solution for managing Kafka. It will be proprietary (we will keep working on the open source UIs of course) but you may want to have a look. We will give free licenses to developers.
More on our site: https://www.landoop.com/
Cheers!
@andmarios Awesome! Looking forward to it!
Thanks!