openanalytics/shinyproxy-operator

Volume Mounts issue

Closed this issue · 5 comments

anoop7181 commented

I am trying to setup volume mounts using Kubernetes patch specifications. Unfortunately after applying it ,I noticed errors in operator pod logs.
I created PV and PVC separately and the setup of Persistent volume and claim looks fine. I tested out this with another pod.

When I refer the same as below , it fails.

kubernetesPodTemplateSpecPatches: |
    - op: add
      path: /spec/volumes
      value:
        - name: "demoplanning"
          azureFile:
            secretName: demo-azure-secret
            shareName: demoplanningdata
            readOnly: false
    - op: add
      path: /spec/containers/0/volumeMounts
      value:
        - mountPath: "/mnt/azure"
          name: "demoplanning"

I do see during shinyproxy pod creation it detecting the volume correctly but below error throws up and shinyproxy is deployed as vanilla installation. Funnily I can access shinyproxy (vanilla installation) but none of customisations like authentication and new containers get applied.

pool-1-thread-1] WARN  e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] Updating latest marker to c28a3f90feed4054d3667bb849cc8e800a33ef15, status: ShinyProxyStatus(instances=[ShinyProxyInstance(hashOfSpec=a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a, isLatestInstance=true), ShinyProxyInstance(hashOfSpec=c28a3f90feed4054d3667bb849cc8e800a33ef15, isLatestInstance=false)])
java.lang.Throwable: null
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController.updateLatestMarker(ShinyProxyController.kt:207)
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController.reconcileSingleShinyProxyInstance(ShinyProxyController.kt:272)
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController$receiveAndHandleEvent$2.invokeSuspend(ShinyProxyController.kt:106)
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController$receiveAndHandleEvent$2.invoke(ShinyProxyController.kt)
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController.receiveAndHandleEvent(ShinyProxyController.kt:117)
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController.run(ShinyProxyController.kt:67)
        at eu.openanalytics.shinyproxyoperator.controller.ShinyProxyController$run$1.invokeSuspend(ShinyProxyController.kt)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:122)
        at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:111)
        at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:308)
        at kotlinx.coroutines.CancellableContinuationImpl.completeResume(CancellableContinuationImpl.kt:395)
        at kotlinx.coroutines.channels.AbstractChannel$ReceiveElement.completeResumeReceive(AbstractChannel.kt:872)
        at kotlinx.coroutines.channels.ArrayChannel.offerInternal(ArrayChannel.kt:83)
        at kotlinx.coroutines.channels.AbstractSendChannel.send(AbstractChannel.kt:133)
        at eu.openanalytics.shinyproxyoperator.controller.ResourceListener.enqueueResource(ResourceListener.kt:78)
        at eu.openanalytics.shinyproxyoperator.controller.ResourceListener$1$onUpdate$2.invokeSuspend(ResourceListener.kt:50)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at eu.openanalytics.shinyproxyoperator.controller.ResourceListener$1.onUpdate(ResourceListener.kt:50)
        at eu.openanalytics.shinyproxyoperator.controller.ResourceListener$1.onUpdate(ResourceListener.kt:42)
        at io.fabric8.kubernetes.client.informers.cache.ProcessorListener$UpdateNotification.handle(ProcessorListener.java:107)
        at io.fabric8.kubernetes.client.informers.cache.ProcessorListener.run(ProcessorListener.java:57)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
14:14:22.523 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/global] Trying to update status (attempt 1/5)
14:14:22.539 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/global] Status successfully updated
14:14:22.539 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 5/6: Ok] [Status/LatestMarker]
14:14:22.540 [ool-13-thread-1] DEBUG e.o.s.controller.ShinyProxyListener  - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Event/Update]
14:14:22.589 [pool-1-thread-1] DEBUG e.o.s.i.skipper.IngressController    - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Component/Ingress] Reconciling
14:14:22.623 [pool-1-thread-1] DEBUG e.o.s.ingress.skipper.IngressFactory - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Component/Ingress] Created sp-shinyproxy-ing-a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a [latest=false]
14:14:22.623 [pool-1-thread-1] DEBUG e.o.s.i.skipper.IngressController    - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Component/Ingress] Reconciling
14:14:22.634 [pool-1-thread-1] DEBUG e.o.s.ingress.skipper.IngressFactory - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Component/Ingress] Created sp-shinyproxy-ing-c28a3f90feed4054d3667bb849cc8e800a33ef15 [latest=true]
14:14:22.634 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 6/6: Ok] [Component/Ingress]
14:14:22.650 [pool-1-thread-1] INFO  e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 0/6: Ok] ReconcileSingleShinyProxy
14:14:22.650 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 1/6: Ok] [Component/ConfigMap]
14:14:22.650 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 2/6: Ok] [Component/ReplicaSet]
14:14:22.650 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 3/6: Ok] [Component/ReplicaSet] ReplicaSet ready
14:14:22.650 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 4/6: Ok] [Component/Service]
14:14:22.650 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 5/6: Ok] [Status/LatestMarker]
14:14:22.656 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 6/6: Ok] [Component/Ingress]
14:14:22.663 [pool-1-thread-1] INFO  e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 0/6: Ok] ReconcileSingleShinyProxy
14:14:22.663 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 1/6: Ok] [Component/ConfigMap]
14:14:22.663 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 2/6: Ok] [Component/ReplicaSet]
14:14:22.663 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 3/6: Ok] [Component/ReplicaSet] ReplicaSet ready
14:14:22.663 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 4/6: Ok] [Component/Service]
14:14:22.664 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 5/6: Ok] [Status/LatestMarker]
14:14:22.669 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/a5f70bedbb2eba4683e6c0a96a4e69777a0d1b2a] [Step 6/6: Ok] [Component/Ingress]
14:14:22.675 [pool-1-thread-1] INFO  e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 0/6: Ok] ReconcileSingleShinyProxy
14:14:22.675 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 1/6: Ok] [Component/ConfigMap]
14:14:22.675 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 2/6: Ok] [Component/ReplicaSet]
14:14:22.675 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 3/6: Ok] [Component/ReplicaSet] ReplicaSet ready
14:14:22.675 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 4/6: Ok] [Component/Service]
14:14:22.675 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 5/6: Ok] [Status/LatestMarker]
14:14:22.690 [pool-1-thread-1] DEBUG e.o.s.c.ShinyProxyController         - [shinyproxy/shinyproxy/c28a3f90feed4054d3667bb849cc8e800a33ef15] [Step 6/6: Ok] [Component/Ingress]
LEDfan commented

Hi @anoop7181

Thanks for your interest in the operator. The error/stacktrace you are getting, is not an error, we purposely added an extra log statement in order to debug a possible bug. However, in Java/Kotlin you'll have to use a fake exception in order to log a stacktrace (see: https://github.com/openanalytics/shinyproxy-operator/blob/develop/src/main/kotlin/eu/openanalytics/shinyproxyoperator/controller/ShinyProxyController.kt#L207 (the actual comment is wrong..)). I understand this is confusing, so I'll adapt the statement before our first release.

Regarding your main issue: the problem is (probably) in your patch. What your patch currently is doing is "at the /spec/volumes path, add an array with a single value name: "demoplanning", ...". In other words, you are replacing the existing volumes array with a new array, which only contains the value you provided in your patch. Therefore, the configmap created and mounted by the Operator is removed from the list. This is the exact reason why you now see that ShinyProxy is using the default configuration. You can adapt your patch, so that it appends your element to the list of volumes:

kubernetesPodTemplateSpecPatches: |
    - op: add
      path: /spec/volumes/-
      value:
          name: "demoplanning"
          azureFile:
            secretName: demo-azure-secret
            shareName: demoplanningdata
            readOnly: false
    - op: add
      path: /spec/containers/0/volumeMounts/-
      value:
         mountPath: "/mnt/azure"
          name: "demoplanning"

I hope this helps!

  • ❤️1
anoop7181 commented

Thank you so much @LEDfan . I copied the earlier syntax from the documentation below https://shinyproxy.io/documentation/configuration/#kubernetes

Now it works and I see the volumes being mounted. Currently these volumes are mounted on the proxy container. Can you please let me know how can I get it mounted on to shiny app container.

anoop7181 commented

Got it working but adding respective Kubernetes-pod-patches template into the container specs. The syntax is different from KubernetesPodTemplateSpecPatches.

For container spec pod template patch , used below syntax
`- id: 01_hello_manifests_espression
container-specs:

  • image: "openanalytics/shinyproxy-demo"
    cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
    port-mapping:
    default: 3838
    kubernetes-pod-patches: |
    • op: add
      path: /spec/containers/0/env/-
      value:
      name: PROXY_ID
      value: "#{proxy.id}"
    • op: add
      path: /spec/volumes
      value:
      • name: "home-dir-pvc-#{proxy.userId}"
        persistentVolumeClaim:
        claimName: "home-dir-pvc-#{proxy.userId}"
    • op: add
      path: /spec/containers/0/volumeMounts
      value:
      • mountPath: "/home/#{proxy.userId}"
        name: "home-dir-pvc-#{proxy.userId}"`
  • 👍1
anoop7181 commented

@LEDfan Thank you for the Operator. It has helped me ease the deployment on AKS.

  • 🎉1
LEDfan commented

Great that you found the correct configuration options! Good to know that the operator works on AKS as well and that it is useful to your.

I'm closing this issue now, feel free to open a new one if you encounter any issue!