apache/airflow-site

If render_template_as_native_obj=True SimpleHttpOperator converts header values to integer automatically

Closed this issue · 1 comments

Apache Airflow version

  • Other Airflow 2 version

What happened?

  • If render_template_as_native_obj=True
  • SimpleHttpOperator converts header values to integer automatically.

headers = {"Content-Type": "application/json", "accept": "application/json", "Accept-Version": "V2", "OTP": "123345"} is interpretted as headers = {"Content-Type": "application/json", "accept": "application/json", "Accept-Version": "V2", "OTP": 123345}

OTP value has to be of string datatype but characters are numbers.

Airflow automatically converts this OTP value to integer whereas the endpoint wants OTP value as string.
This is the error from the endpoint, this is to explain the error I am getting from airflow
requests.exceptions.InvalidHeader: Value for header {OTP: 123345} must be of type str or bytes, not <class 'int'>

If render_template_as_native_obj=False, then SimpleHttpOperator works just fine but I want to use render_template_as_native_obj=True for other requirements.

What you think should happen instead
Expected:
Please suggest a way to use render_template_as_native_obj=True and be able to send numbers as string in headers of SimpleHttpOperators

How to reproduce

with DAG('mydag', schedule_interval='@daily', default_args=default_args, catchup=False, render_template_as_native_obj=True) as dag:
      task = SimpleHttpOperator(
              task_id = 'taskname',
              method = "POST",
              http_conn_id = "host",
              endpoint = "/endpoint",
              headers = {"Content-Type": "application/json", "accept": "application/json", "Accept-Version": "V2", "OTP": "123345"}',
              data = json.dumps({"csr": "{{ ti.xcom_pull(task_ids='task1', key='return_value').replace(' ', '')}}" }),
              response_check = lambda response: _handle_response_check(response),
              response_filter = lambda response: {"id": response.json()["id"], "token": response.json()["token"], "secret": response.json()["secret"]}
          )

This is how the task looks like.

Operating System
Ubuntu 20.04

Versions of Apache Airflow Providers
apache/airflow:2.3.0 as base image and added few python libraries and stored in a private docker repository.
Using this image with helm.

Deployment
Official Apache Airflow Helm Chart

will be handled in apache/airflow#26149