spring-cloud/spring-cloud-deployer

Node Selector doesn't get applied on K8s platforms

juanpablo-santos opened this issue · 4 comments

juanpablo-santos commented

Description:
per title, Node Selector configuration option doesn't get applied on K8s platforms, neither by setting spring.cloud.dataflow.task.platform.kubernetes.XXXX.deployment.nodeSelector nor by applying the configuration on task execution, through the UI (the deployer.XXXXXX.kubernetes.node-selector=NAME:VALUE)

Release versions:

{
  "versions": {
    "implementation": {
      "name": "spring-cloud-dataflow-server",
      "version": "2.11.2"
    },
    "core": {
      "name": "Spring Cloud Data Flow Core",
      "version": "2.11.2"
    },
    "dashboard": {
      "name": "Spring Cloud Dataflow UI",
      "version": "3.4.2"
    },
    "shell": {
      "name": "Spring Cloud Data Flow Shell",
      "version": "2.11.2",
      "url": "https://repo.maven.apache.org/maven2/org/springframework/cloud/spring-cloud-dataflow-shell/2.11.2/spring-cloud-dataflow-shell-2.11.2.jar"
    }
  },
  "features": {
    "streams": true,
    "tasks": true,
    "schedules": true,
    "monitoringDashboardType": "GRAFANA"
  },
  "runtimeEnvironment": {
    "appDeployer": {
      "deployerImplementationVersion": "2.11.2",
      "deployerName": "Spring Cloud Skipper Server",
      "deployerSpiVersion": "2.11.2",
      "javaVersion": "17.0.9",
      "platformApiVersion": "",
      "platformClientVersion": "",
      "platformHostVersion": "",
      "platformSpecificInfo": {
        "default": "kubernetes"
      },
      "platformType": "Skipper Managed",
      "springBootVersion": "2.7.18",
      "springVersion": "5.3.31"
    },
    "taskLaunchers": [
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      },
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      },
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      },
      {
        "deployerImplementationVersion": "unknown",
        "deployerName": "KubernetesTaskLauncher",
        "deployerSpiVersion": "unknown",
        "javaVersion": "17.0.9",
        "platformApiVersion": "v1",
        "platformClientVersion": "unknown",
        "platformHostVersion": "unknown",
        "platformSpecificInfo": {
          "namespace": "scdf",
          "master-url": "XXXXXXXXX"
        },
        "platformType": "Kubernetes",
        "springBootVersion": "2.7.18",
        "springVersion": "5.3.31"
      }
    ]
  },
  "monitoringDashboardInfo": {
    "url": "https://grafana.XXXXXXXXX.dom",
    "source": "default-scdf-source",
    "refreshInterval": 15
  },
  "security": {
    "isAuthentication": true,
    "isAuthenticated": true,
    "username": "YYYYYY",
    "roles": [
      "ROLE_CREATE",
      "ROLE_DEPLOY",
      "ROLE_DESTROY",
      "ROLE_MANAGE",
      "ROLE_MODIFY",
      "ROLE_SCHEDULE",
      "ROLE_VIEW"
    ]
  },
  "git": {
    "commit": "86b53e0"
  }
}

Custom apps:
N/A, issue is reproduced with any task execution.

Steps to reproduce:
Neither setting spring.cloud.dataflow.task.platform.kubernetes.XXXX.deployment.nodeSelector: "NAME:VALUE" nor by applying the configuration on task execution, through the UI (the deployer.XXXXXX.kubernetes.node-selector=NAME:VALUE.

Screenshots:
pod-describe
task-exec-config

Additional context:
Saw PR spring-cloud/spring-cloud-dataflow#301, the configuration provided there also doesn't work

cppwfs commented

@juanpablo-santos Thank you for reporting this issue:
A work around is to set the following property for node selectors: deployer.<appname>.kubernetes.deployment.nodeSelector=foo:bar

We will work on resolving this bug.

juanpablo-santos commented

Hi @cppwfs! thanks a lot for the workaround :-)

I did try the property, but instead of using nodeSelector I did use node-selector (as generated by the UI), thought they were interchangeable and didn't thought on using nodeSelector. In any case, the workaround is more than enough for us until the issue is fixed.

cheers,
juan pablo

corneil commented

Thanks @juanpablo-santos for doing the legwork.
We will be fixing that one since was expecting a property named with nodeSelector instead of allowing node-selector.

juanpablo-santos commented

Hi @corneil! saw the associated PR, would that cover the platform property as well? i.e., spring.cloud.dataflow.task.platform.kubernetes.XXXX.deployment.nodeSelector also doesn't get assigned

thanks for looking into this so fast :-)