spring-cloud/spring-cloud-dataflow

Not able to add IAM role

Opened this issue · 6 comments

Description:
How can we add the pod annotaions, I need to add the IAM roles for all pods which gets launched by SCDF.

Release versions:

we are using "bitnami/spring-cloud-dataflow:2.11.5-debian-12-r2"

Screenshots:
Sample screen shot is below
@@
Screenshot 2024-12-09 at 1 18 15 PM

Additional context

Below is our values.yaml file

server:
image:
registry: docker.io
repository: bitnami/spring-cloud-dataflow
tag: 2.11.5-debian-12-r2
digest: ""
pullPolicy: IfNotPresent
pullSecrets: []
debug: false
composedTaskRunner:
image:
registry: docker.io
repository: bitnami/spring-cloud-dataflow-composed-task-runner
tag: 2.11.5-debian-12-r2
digest: ""
configuration:
streamingEnabled: false
batchEnabled: true
accountName: default
trustK8sCerts: false
containerPorts:
http: 8080
jdwp: 5005
replicaCount: 1
updateStrategy:
type: RollingUpdate
startupProbe:
enabled: false
initialDelaySeconds: 120
timeoutSeconds: 1
periodSeconds: 20
failureThreshold: 6
successThreshold: 1
livenessProbe:
enabled: true
initialDelaySeconds: 120
timeoutSeconds: 1
periodSeconds: 20
failureThreshold: 6
successThreshold: 1
readinessProbe:
enabled: true
initialDelaySeconds: 120
timeoutSeconds: 1
periodSeconds: 20
failureThreshold: 6
successThreshold: 1
networkPolicy:
enabled: false
allowExternal: false
allowExternalEgress: false
service:
type: ClusterIP
ports:
http: 8080
ingress:
enabled: true
path: /
pathType: ImplementationSpecific
hostname: "xyz.com"
pdb:
create: false
minAvailable: ""
maxUnavailable: ""
pdb:
create: false
skipper:
enabled: false
rabbitmq:
enabled: false
mariadb:
enabled: false
metrics:
enabled: false
pdb:
create: false
externalDatabase:
host: "{{RDS-endpoint}}.rds.amazonaws.com
driver: com.mysql.cj.jdbc.Driver
dataflow:
url: "{Database url}"
username:
password:

I believe there are podAnnotations in the Bitnami Helm values schema.
You can set different for skipper stream deployments and dataflow task launches.

We are using this as below as we didn't find the dataflow task launches and it is not working, could you please suggest here

server:
podAnnotations:
iam.amazonaws.com/role: ' '
service:
annotations:
iam.amazonaws.com/role: ' '

@corneil , will you please help here

If you are trying to add pod annotations to your application you can add them using the following instructions:https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#_annotations

Hi @cppwfs , we have tried that , its not working. Please check the below configuration for reference and Thanks in advance.

deployer:
dsp:
kubernetes:
podAnnotations: iam.amazonaws.com/role:''
serviceAnnotations: iam.amazonaws.com/role:''
jobAnnotations: iam.amazonaws.com/role:''

deployer:
default:
kubernetes:
podAnnotations: iam.amazonaws.com/role:''
serviceAnnotations: iam.amazonaws.com/role:''
jobAnnotations: iam.amazonaws.com/role:''

It seems like the Bitnami Helm chart doesn't provide for adding a generic block of configuration for dataflow or skipper like we do in our Carvel deployment. The plain kubernetes deployment also provides for extending the spring.cloud.dataflow or spring.cloud.skipper configuration with YAML or environment variables. https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-kubernetes
It seems like you will have to use extraEnvVarsCM or extraEnvVars to provide the full properties like

server:
  extraEnvVars:
    - name: SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_PODANNOTATIONS
      value: "iam.amazonaws.com/role: ''"
    - name: SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_JOBANNOTATIONS
      value: "iam.amazonaws.com/role: ''""