astronomer/dag-factory

How to access Airflow variables and parameters passed via REST API

Opened this issue · 2 comments

I have a use case where I am accessing Airflow's variable using Variable.get(). How can I achieve this in yaml file.

Also, how can I access parameters (params.app_name, params.namespace etc) which I am passing via REST API as follows:

{ "conf": { "app_name":"app namet", "region":"", "namespace": "", "default_path": "", } }

env_variables = Variable.get("env_variables", deserialize_json=True)
fetch_yaml = BashOperator(
        bash_command="export {{var.json.env_variables['kube_config'] && kubectl get sparkapplications {{ params.app_name}} -n {{params.namespace}}",
        task_id="fetch_yaml"
    )

Airflow variables can be accessed as below

within yaml specify any attribute with the value as shown below. Note the quotes, some of them are two single quotes.
ENV: '{{ var.value.get(''ENVIRONMENT'', ''no-mans-land'') }}'

I am also looking at how to access params though, which is typically as

DAG(params={...}), however specifying these with in yaml does not resolve.

YAML

params:
    key: value

comes out as below when printing ... print(vars(context.get('dag')))

'params': {}

Reference : https://github.com/apache/airflow/blob/main/airflow/models/dag.py#L229

PR submitted.

#124