ForgeRock/forgeops

blank fqdn redirect on custom domain setting via helm

dpmerron-ltd opened this issue · 19 comments

dpmerron-ltd commented

Hi,

We are using helm to install OpenAM - when using the default .forgeops.com domain (using local host file to fake dns) we are able to see the OpenAM XUI login portal.

When changing the domain to .<my own fqdn> we also get a redirect to the XUI portal path but the redirected path is stripped of our fqdn and therefore fails.

To clarify, instead of openam.default.<myfqdn>/openam/XUI//#login I get redirected to just openam/XUI/#login/.

I have updated the following values files with my domain also including the prefixed .:

helm/amster/values.yaml
helm/frconfig/values.yaml
helm/openam/values.yaml

Happy to provide more info if required?

Thanks

wstrange commented

Can you try to set the BASE URL service in the AM configuration. Go to the root realm -> services -> Add base URL service. Add your fqdn to that

dpmerron-ltd commented

We are currently using the public forgeops-init repo as our config just to get the product up. Does this mean we have to run this using the forgeops.com domain name before we can modify this to use our own fqdn.

Dan

wstrange commented

We install this on several different domains - so you should be able to bring it up on a non forgeops.com domain. Are you using the default "empty-import" config (which is basically just an install)?

You will eventually want to fork / clone your own forgeops-init repo.

I'd also try something like helm template amster with your values settings to be sure that all the references to forgeops.com are replaced with your own FQDN

dpmerron-ltd commented 
# This is a YAML-formatted file.


config:
  # The default name that products use when looking for the configmap and secret is `frconfig`.
  # To create per-product configurations, you can deploy multiple instances of this chart
  # using a different name. The corresponding product chart also need to override config.name.
  # See README.md for more information.
  name: frconfig

git:
  repo: https://github.com/ForgeRock/forgeops-init.git
  branch: master

domain: .mysite.co.uk

This is the values file for the frconfig chart. Does this look right?

wstrange commented

The one thing that stands out is the 3 part domain (.mysite.co.uk). Can you try with a 2 part domain (acme.com) to see if that has any impact. We might have an issue here.

wstrange commented

Also - can you post your ingress.

kubectl get ing openam -o yaml

I suspect the ingress is not properly formed. You may have to explicitly set it in the AM chart

c-mccutcheon commented
c-mccutcheon commented

@wstrange here is our generated ingress, doesn't look any different to the one which creates when we use the .forgeops.com domain as far as i can see

kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/cors-allow-credentials: "false"
    nginx.ingress.kubernetes.io/cors-allow-headers: authorization,x-requested-with
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT,GET,POST,HEAD,OPTIONS,DELETE
    nginx.ingress.kubernetes.io/cors-allow-origin: '*'
    nginx.ingress.kubernetes.io/cors-max-age: "600"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/session-cookie-hash: sha1
    nginx.ingress.kubernetes.io/session-cookie-name: route
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  creationTimestamp: 2018-12-04T10:42:19Z
  generation: 3
  labels:
    app: open-access-management-openam
    chart: openam-6.5.21292
    heritage: Tiller
    release: open-access-management
    vendor: forgerock
  name: openam
  namespace: default
  resourceVersion: "21729"
  selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/openam
  uid: 46749fed-f7b1-11e8-a276-6e0d607a9068
spec:
  rules:
  - host: openam.default.our.domain.co.uk
    http:
      paths:
      - backend:
          serviceName: openam
          servicePort: 80
        path: /openam
status:
  loadBalancer:
    ingress:
    - ip: xxx.xx.xx.xxx

IP is just masked by me, but is correct. Only real difference as seen is the host rule is simply different to when using .forgeops.com.

One thing I will add - I built the 6.5 version of the openam, amster and ds containers, and we maintain our own charts which are forked from this repository as per the DevOps guide (we are a commercial customer) - I didn't think this would cause any major issue (given it works with your helm defaults) but just in case.

Cheers

wstrange commented

That fqdn has 6 parts: openam.default.our.domain.co.uk
I think you installed amster settting the domain to .domain.co.uk. So I am guessing there is a mismatch between what am things is the external FQDN, and what the ingress is configured for. Run
helm template amster and look at the configmap for amster. What does the load balancer URL look like?

In 6.5, If you are using our autoexpanding template for ingress (really a convenience feature), the pattern is:

login.{namespace}{domain}

Note that if you are using 6.5 images, we no longer use /openam as the context . AM is installed at the context root /

If you are using 6.5 images it would be best to rebase on master.

c-mccutcheon commented
dgoldssfo commented

Hi, Clinton,

It's in the new Release Notes: https://backstage.forgerock.com/docs/platform/6.5/release-notes

David

wstrange commented

I'm a little confused as to your current state, but here are things to double check:

  • Make sure the value of lbPrimaryUrl in the amster-config configmap exactly matches the ingress hostname.
  • If you are using the 6.5 docker images that we publish on bintray, you must also use the corresponding helm charts for 6.5. The FQDN for AM has changed to `login.{namespace}{domain}/ . Note there is no /openam anymore. The ingress definition has been updated to reflect this.
  • Watch out for DNS issues when using fake host names. I dont think this is your issue, but I have seen problems with this in the past
  • You may have to add the base url service to the root realm. When doing an empty-import (i.e. just install) this should not be required- but it won't hurt.
  • Make sure your clear any browser cookies from previous attempts.
c-mccutcheon commented

Thanks @wstrange @dgoldssfo - we are going to try a fresh build up of 6.5.0 following the newer guide and see if we get the same issue.

c-mccutcheon commented

@wstrange - struggling with the 6.5.0 release, even following the DevOps guide as above we seem to encounter many errors and DS container crashes routinely now.

Does (or should, rather) this guide work end to end? CTS/User/Config stores using DS seem to have a particular problem, which means OpenAM doesn't boot - but Amster seems to load up OK now.

Error is as follows

Server id is 10
Data volume contains existing data
Updating the directory manager password
An error occurred while attempting to initialize server components to run the
tool: An error occurred while trying to load the Directory Server
configuration:  Unable to load the configuration-enabled schema:
FileNotFoundException: 02-config.ldif (No such file or directory)
(FileInputStream.java:-2 FileInputStream.java:195 FileInputStream.java:138
FileReader.java:72 ConfigurationHandler.java:958 ConfigurationHandler.java:227
ConfigurationHandler.java:211 DirectoryServer.java:1189
DirectoryServer.java:1176 DirectoryServer.java:672 DirectoryServer.java:628
EncodePassword.java:229 Utils.java:972 Utils.java:949 EncodePassword.java:74)
(id=org.opends.messages.tool-21)
head: cannot open 'data/db/rootUser/rootUser.ldif' for reading: No such file or directory
mv: cannot move '/tmp/pw' to 'data/db/rootUser/rootUser.ldif': No such file or directory
Updating the monitor user password
An error occurred while attempting to initialize server components to run the
tool: An error occurred while trying to load the Directory Server
configuration:  Unable to load the configuration-enabled schema:
FileNotFoundException: 02-config.ldif (No such file or directory)
(FileInputStream.java:-2 FileInputStream.java:195 FileInputStream.java:138
FileReader.java:72 ConfigurationHandler.java:958 ConfigurationHandler.java:227
ConfigurationHandler.java:211 DirectoryServer.java:1189
DirectoryServer.java:1176 DirectoryServer.java:672 DirectoryServer.java:628
EncodePassword.java:229 Utils.java:972 Utils.java:949 EncodePassword.java:74)
(id=org.opends.messages.tool-21)
head: cannot open 'data/db/monitorUser/monitorUser.ldif' for reading: No such file or directory
mv: cannot move '/tmp/pw' to 'data/db/monitorUser/monitorUser.ldif': No such file or directory
Starting DS
Server id 10
An error occurred while attempting to bootstrap the Directory Server: Unable to load the configuration-enabled schema: FileNotFoundException: 02-config.ldif (No such file or directory) (FileInputStream.java:-2 FileInputStream.java:195 FileInputStream.java:138 FileReader.java:72 ConfigurationHandler.java:958 ConfigurationHandler.java:227 ConfigurationHandler.java:211 DirectoryServer.java:1189 DirectoryServer.java:4023)
[06/Dec/2018:14:55:09 +0000] category=CORE severity=NOTICE msgID=139 msg=The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerShutdown, alert ID org.opends.messages.core-141): The Directory Server has started the shutdown process. The shutdown was initiated by an instance of class org.opends.server.core.DirectoryServerShutdownHook and the reason provided for the shutdown was The Directory Server shutdown hook detected that the JVM is shutting down. This generally indicates that JVM received an external request to stop (e.g., through a kill signal)
[06/Dec/2018:14:55:10 +0000] category=CORE severity=NOTICE msgID=203 msg=The Directory Server is now stopped
[INFO  tini (1)] Spawned child process './bin/start-ds' with pid '80'
[INFO  tini (1)] Main child exited normally (with status '1')````
wstrange commented

FileNotFoundException: 02-config.ldif (No such file or directory) is not expected.

What platform are you testing on? Are you sure the persistent volume is correctly mounted on /opt/opendj/data?

[This has been tested on GKE, EKS and minikube]

c-mccutcheon commented

We are on Azure AKS - installation ended up working for 6.0.0 with the exception of the weird URL issue that we opened above.

Just checking the persistent volumes and for cts-store (and others) the volume claims / mounts appear to be correct (see below):

"name": "db",
           "mountPath": "/opt/opendj/data"
         },
wstrange commented

That error would seem to imply that the proto db/ files did not get copied to the persistent store, or that the persistent store is not properly mounted.

Given that we have strayed a bit from the original bug, I'd like to close this issue ;-)

Can I get you to file a ticket with support - I think it will be easier than using github issues.

wstrange commented

I'll add: We are planning Azure AKS support - so we are very interested in getting any issues resolved

c-mccutcheon commented

no problem @wstrange - feel free to close this one.